Browse
Abstract
Extracting valuable information from large sets of diverse meteorological data is a time-intensive process. Machine-learning methods can help to improve both speed and accuracy of this process. Specifically, deep-learning image-segmentation models using the U-Net structure perform faster and can identify areas that are missed by more restrictive approaches, such as expert hand-labeling and a priori heuristic methods. This paper discusses four different state-of-the-art U-Net models designed for detection of tropical and extratropical cyclone regions of interest (ROI) from two separate input sources: total precipitable water output from the Global Forecast System (GFS) model and water vapor radiance images from the Geostationary Operational Environmental Satellite (GOES). These models are referred to as International Best Track Archive for Climate Stewardship (IBTrACS)-GFS, Heuristic-GFS, IBTrACS-GOES, and Heuristic-GOES. All four U-Nets are fast information extraction tools and perform with an ROI detection accuracy ranging from 80% to 99%. These are additionally evaluated with the Dice and Tversky intersection-over-union (IoU) metrics, having Dice coefficient scores ranging from 0.51 to 0.76 and Tversky coefficients ranging from 0.56 to 0.74. The extratropical cyclone U-Net model performed 3 times as fast as the comparable heuristic model used to detect the same ROI. The U-Nets were specifically selected for their capabilities in detecting cyclone ROI beyond the scope of the training labels. These machine-learning models identified more ambiguous and active ROI missed by the heuristic model and hand-labeling methods that are commonly used in generating real-time weather alerts, having a potentially direct impact on public safety.
Abstract
Extracting valuable information from large sets of diverse meteorological data is a time-intensive process. Machine-learning methods can help to improve both speed and accuracy of this process. Specifically, deep-learning image-segmentation models using the U-Net structure perform faster and can identify areas that are missed by more restrictive approaches, such as expert hand-labeling and a priori heuristic methods. This paper discusses four different state-of-the-art U-Net models designed for detection of tropical and extratropical cyclone regions of interest (ROI) from two separate input sources: total precipitable water output from the Global Forecast System (GFS) model and water vapor radiance images from the Geostationary Operational Environmental Satellite (GOES). These models are referred to as International Best Track Archive for Climate Stewardship (IBTrACS)-GFS, Heuristic-GFS, IBTrACS-GOES, and Heuristic-GOES. All four U-Nets are fast information extraction tools and perform with an ROI detection accuracy ranging from 80% to 99%. These are additionally evaluated with the Dice and Tversky intersection-over-union (IoU) metrics, having Dice coefficient scores ranging from 0.51 to 0.76 and Tversky coefficients ranging from 0.56 to 0.74. The extratropical cyclone U-Net model performed 3 times as fast as the comparable heuristic model used to detect the same ROI. The U-Nets were specifically selected for their capabilities in detecting cyclone ROI beyond the scope of the training labels. These machine-learning models identified more ambiguous and active ROI missed by the heuristic model and hand-labeling methods that are commonly used in generating real-time weather alerts, having a potentially direct impact on public safety.
Abstract
The method of neural networks (aka deep learning) has opened up many new opportunities to utilize remotely sensed images in meteorology. Common applications include image classification, e.g., to determine whether an image contains a tropical cyclone, and image-to-image translation, e.g., to emulate radar imagery for satellites that only have passive channels. However, there are yet many open questions regarding the use of neural networks for working with meteorological images, such as best practices for evaluation, tuning, and interpretation. This article highlights several strategies and practical considerations for neural network development that have not yet received much attention in the meteorological community, such as the concept of receptive fields, underutilized meteorological performance measures, and methods for neural network interpretation, such as synthetic experiments and layer-wise relevance propagation. We also consider the process of neural network interpretation as a whole, recognizing it as an iterative meteorologist-driven discovery process that builds on experimental design and hypothesis generation and testing. Finally, while most work on neural network interpretation in meteorology has so far focused on networks for image classification tasks, we expand the focus to also include networks for image-to-image translation.
Abstract
The method of neural networks (aka deep learning) has opened up many new opportunities to utilize remotely sensed images in meteorology. Common applications include image classification, e.g., to determine whether an image contains a tropical cyclone, and image-to-image translation, e.g., to emulate radar imagery for satellites that only have passive channels. However, there are yet many open questions regarding the use of neural networks for working with meteorological images, such as best practices for evaluation, tuning, and interpretation. This article highlights several strategies and practical considerations for neural network development that have not yet received much attention in the meteorological community, such as the concept of receptive fields, underutilized meteorological performance measures, and methods for neural network interpretation, such as synthetic experiments and layer-wise relevance propagation. We also consider the process of neural network interpretation as a whole, recognizing it as an iterative meteorologist-driven discovery process that builds on experimental design and hypothesis generation and testing. Finally, while most work on neural network interpretation in meteorology has so far focused on networks for image classification tasks, we expand the focus to also include networks for image-to-image translation.
Abstract
Artificial intelligence (AI) techniques have had significant recent successes in multiple fields. These fields and the fields of satellite remote sensing and NWP share the same fundamental underlying needs, including signal and image processing, quality control mechanisms, pattern recognition, data fusion, forward and inverse problems, and prediction. Thus, modern AI in general and machine learning (ML) in particular can be positively disruptive and transformational change agents in the fields of satellite remote sensing and NWP by augmenting, and in some cases replacing, elements of the traditional remote sensing, assimilation, and modeling tools. And change is needed to meet the increasing challenges of Big Data, advanced models and applications, and user demands. Future developments, for example, SmallSats and the Internet of Things, will continue the explosion of new environmental data. ML models are highly efficient and in some cases more accurate because of their flexibility to accommodate nonlinearity and/or non-Gaussianity. With that efficiency, ML can help to address the demands put on environmental products for higher accuracy, for higher resolution—spatial, temporal, and vertical, for enhanced conventional medium-range forecasts, for outlooks and predictions on subseasonal to seasonal time scales, and for improvements in the process of issuing advisories and warnings. Using examples from satellite remote sensing and NWP, it is illustrated how ML can accelerate the pace of improvement in environmental data exploitation and weather prediction—first, by complementing existing systems, and second, where appropriate, as an alternative to some components of the NWP processing chain from observations to forecasts.
Abstract
Artificial intelligence (AI) techniques have had significant recent successes in multiple fields. These fields and the fields of satellite remote sensing and NWP share the same fundamental underlying needs, including signal and image processing, quality control mechanisms, pattern recognition, data fusion, forward and inverse problems, and prediction. Thus, modern AI in general and machine learning (ML) in particular can be positively disruptive and transformational change agents in the fields of satellite remote sensing and NWP by augmenting, and in some cases replacing, elements of the traditional remote sensing, assimilation, and modeling tools. And change is needed to meet the increasing challenges of Big Data, advanced models and applications, and user demands. Future developments, for example, SmallSats and the Internet of Things, will continue the explosion of new environmental data. ML models are highly efficient and in some cases more accurate because of their flexibility to accommodate nonlinearity and/or non-Gaussianity. With that efficiency, ML can help to address the demands put on environmental products for higher accuracy, for higher resolution—spatial, temporal, and vertical, for enhanced conventional medium-range forecasts, for outlooks and predictions on subseasonal to seasonal time scales, and for improvements in the process of issuing advisories and warnings. Using examples from satellite remote sensing and NWP, it is illustrated how ML can accelerate the pace of improvement in environmental data exploitation and weather prediction—first, by complementing existing systems, and second, where appropriate, as an alternative to some components of the NWP processing chain from observations to forecasts.
Abstract
Most ensembles suffer from underdispersion and systematic biases. One way to correct for these shortcomings is via machine learning (ML), which is advantageous due to its ability to identify and correct nonlinear biases. This study uses a single random forest (RF) to calibrate next-day (i.e., 12–36-h lead time) probabilistic precipitation forecasts over the contiguous United States (CONUS) from the Short-Range Ensemble Forecast System (SREF) with 16-km grid spacing and the High-Resolution Ensemble Forecast version 2 (HREFv2) with 3-km grid spacing. Random forest forecast probabilities (RFFPs) from each ensemble are compared against raw ensemble probabilities over 496 days from April 2017 to November 2018 using 16-fold cross validation. RFFPs are also compared against spatially smoothed ensemble probabilities since the raw SREF and HREFv2 probabilities are overconfident and undersample the true forecast probability density function. Probabilistic precipitation forecasts are evaluated at four precipitation thresholds ranging from 0.1 to 3 in. In general, RFFPs are found to have better forecast reliability and resolution, fewer spatial biases, and significantly greater Brier skill scores and areas under the relative operating characteristic curve compared to corresponding raw and spatially smoothed ensemble probabilities. The RFFPs perform best at the lower thresholds, which have a greater observed climatological frequency. Additionally, the RF-based postprocessing technique benefits the SREF more than the HREFv2, likely because the raw SREF forecasts contain more systematic biases than those from the raw HREFv2. It is concluded that the RFFPs provide a convenient, skillful summary of calibrated ensemble output and are computationally feasible to implement in real time. Advantages and disadvantages of ML-based postprocessing techniques are discussed.
Abstract
Most ensembles suffer from underdispersion and systematic biases. One way to correct for these shortcomings is via machine learning (ML), which is advantageous due to its ability to identify and correct nonlinear biases. This study uses a single random forest (RF) to calibrate next-day (i.e., 12–36-h lead time) probabilistic precipitation forecasts over the contiguous United States (CONUS) from the Short-Range Ensemble Forecast System (SREF) with 16-km grid spacing and the High-Resolution Ensemble Forecast version 2 (HREFv2) with 3-km grid spacing. Random forest forecast probabilities (RFFPs) from each ensemble are compared against raw ensemble probabilities over 496 days from April 2017 to November 2018 using 16-fold cross validation. RFFPs are also compared against spatially smoothed ensemble probabilities since the raw SREF and HREFv2 probabilities are overconfident and undersample the true forecast probability density function. Probabilistic precipitation forecasts are evaluated at four precipitation thresholds ranging from 0.1 to 3 in. In general, RFFPs are found to have better forecast reliability and resolution, fewer spatial biases, and significantly greater Brier skill scores and areas under the relative operating characteristic curve compared to corresponding raw and spatially smoothed ensemble probabilities. The RFFPs perform best at the lower thresholds, which have a greater observed climatological frequency. Additionally, the RF-based postprocessing technique benefits the SREF more than the HREFv2, likely because the raw SREF forecasts contain more systematic biases than those from the raw HREFv2. It is concluded that the RFFPs provide a convenient, skillful summary of calibrated ensemble output and are computationally feasible to implement in real time. Advantages and disadvantages of ML-based postprocessing techniques are discussed.
Abstract
This paper synthesizes multiple methods for machine learning (ML) model interpretation and visualization (MIV) focusing on meteorological applications. ML has recently exploded in popularity in many fields, including meteorology. Although ML has been successful in meteorology, it has not been as widely accepted, primarily due to the perception that ML models are “black boxes,” meaning the ML methods are thought to take inputs and provide outputs but not to yield physically interpretable information to the user. This paper introduces and demonstrates multiple MIV techniques for both traditional ML and deep learning, to enable meteorologists to understand what ML models have learned. We discuss permutation-based predictor importance, forward and backward selection, saliency maps, class-activation maps, backward optimization, and novelty detection. We apply these methods at multiple spatiotemporal scales to tornado, hail, winter precipitation type, and convective-storm mode. By analyzing such a wide variety of applications, we intend for this work to demystify the black box of ML, offer insight in applying MIV techniques, and serve as a MIV toolbox for meteorologists and other physical scientists.
Abstract
This paper synthesizes multiple methods for machine learning (ML) model interpretation and visualization (MIV) focusing on meteorological applications. ML has recently exploded in popularity in many fields, including meteorology. Although ML has been successful in meteorology, it has not been as widely accepted, primarily due to the perception that ML models are “black boxes,” meaning the ML methods are thought to take inputs and provide outputs but not to yield physically interpretable information to the user. This paper introduces and demonstrates multiple MIV techniques for both traditional ML and deep learning, to enable meteorologists to understand what ML models have learned. We discuss permutation-based predictor importance, forward and backward selection, saliency maps, class-activation maps, backward optimization, and novelty detection. We apply these methods at multiple spatiotemporal scales to tornado, hail, winter precipitation type, and convective-storm mode. By analyzing such a wide variety of applications, we intend for this work to demystify the black box of ML, offer insight in applying MIV techniques, and serve as a MIV toolbox for meteorologists and other physical scientists.
Abstract
A deep learning convolutional neural network model is used to explore the possibilities of estimating tropical cyclone (TC) intensity from satellite images in the 37- and 85–92-GHz bands. The model, called “DeepMicroNet,” has unique properties such as a probabilistic output, the ability to operate from partial scans, and resiliency to imprecise TC center fixes. The 85–92-GHz band is the more influential data source in the model, with 37 GHz adding a marginal benefit. Training the model on global best track intensities produces model estimates precise enough to replicate known best track intensity biases when compared to aircraft reconnaissance observations. Model root-mean-square error (RMSE) is 14.3 kt (1 kt ≈ 0.5144 m s−1) compared to two years of independent best track records, but this improves to an RMSE of 10.6 kt when compared to the higher-standard aircraft reconnaissance-aided best track dataset, and to 9.6 kt compared to the reconnaissance-aided best track when using the higher-resolution TRMM TMI and Aqua AMSR-E microwave observations only. A shortage of training and independent testing data for category 5 TCs leaves the results at this intensity range inconclusive. Based on this initial study, the application of deep learning to TC intensity analysis holds tremendous promise for further development with more advanced methodologies and expanded training datasets.
Abstract
A deep learning convolutional neural network model is used to explore the possibilities of estimating tropical cyclone (TC) intensity from satellite images in the 37- and 85–92-GHz bands. The model, called “DeepMicroNet,” has unique properties such as a probabilistic output, the ability to operate from partial scans, and resiliency to imprecise TC center fixes. The 85–92-GHz band is the more influential data source in the model, with 37 GHz adding a marginal benefit. Training the model on global best track intensities produces model estimates precise enough to replicate known best track intensity biases when compared to aircraft reconnaissance observations. Model root-mean-square error (RMSE) is 14.3 kt (1 kt ≈ 0.5144 m s−1) compared to two years of independent best track records, but this improves to an RMSE of 10.6 kt when compared to the higher-standard aircraft reconnaissance-aided best track dataset, and to 9.6 kt compared to the reconnaissance-aided best track when using the higher-resolution TRMM TMI and Aqua AMSR-E microwave observations only. A shortage of training and independent testing data for category 5 TCs leaves the results at this intensity range inconclusive. Based on this initial study, the application of deep learning to TC intensity analysis holds tremendous promise for further development with more advanced methodologies and expanded training datasets.
Abstract
This study compares the performance of Global Ensemble Forecast System (GEFS) and European Centre for Medium-Range Weather Forecasts (ECMWF) precipitation ensemble forecasts in Brazil and evaluates different analog-based methods and a logistic regression method for postprocessing the GEFS forecasts. The numerical weather prediction (NWP) forecasts were evaluated against the Physical Science Division South America Daily Gridded Precipitation dataset using both deterministic and probabilistic forecasting evaluation metrics. The results show that the ensemble precipitation forecasts performed commonly well in the east and poorly in the northwest of Brazil, independent of the models and the postprocessing methods. While the raw ECMWF forecasts performed better than the raw GEFS forecasts, analog-based GEFS forecasts were more skillful and reliable than both raw ECMWF and GEFS forecasts. The choice of a specific postprocessing strategy had less impact on the performance than the postprocessing itself. Nonetheless, forecasts produced with different analog-based postprocessing strategies were significantly different and were more skillful and as reliable and sharp as forecasts produced with the logistic regression method. The approach considering the logarithm of current and past reforecasts as the measure of closeness between analogs was identified as the best strategy. The results also indicate that the postprocessing using analog methods with long-term reforecast archive improved raw GEFS precipitation forecasting skill more than using logistic regression with short-term reforecast archive. In particular, the postprocessing dramatically improves the GEFS precipitation forecasts when the forecasting skill is low or below zero.
Abstract
This study compares the performance of Global Ensemble Forecast System (GEFS) and European Centre for Medium-Range Weather Forecasts (ECMWF) precipitation ensemble forecasts in Brazil and evaluates different analog-based methods and a logistic regression method for postprocessing the GEFS forecasts. The numerical weather prediction (NWP) forecasts were evaluated against the Physical Science Division South America Daily Gridded Precipitation dataset using both deterministic and probabilistic forecasting evaluation metrics. The results show that the ensemble precipitation forecasts performed commonly well in the east and poorly in the northwest of Brazil, independent of the models and the postprocessing methods. While the raw ECMWF forecasts performed better than the raw GEFS forecasts, analog-based GEFS forecasts were more skillful and reliable than both raw ECMWF and GEFS forecasts. The choice of a specific postprocessing strategy had less impact on the performance than the postprocessing itself. Nonetheless, forecasts produced with different analog-based postprocessing strategies were significantly different and were more skillful and as reliable and sharp as forecasts produced with the logistic regression method. The approach considering the logarithm of current and past reforecasts as the measure of closeness between analogs was identified as the best strategy. The results also indicate that the postprocessing using analog methods with long-term reforecast archive improved raw GEFS precipitation forecasting skill more than using logistic regression with short-term reforecast archive. In particular, the postprocessing dramatically improves the GEFS precipitation forecasts when the forecasting skill is low or below zero.
Nonlinear Wave Ensemble Averaging in the Gulf of Mexico Using Neural NetworksAbstract
Artificial neural networks (ANNs) applied to nonlinear wave ensemble averaging are developed and studied for Gulf of Mexico simulations. It is an approach that expands the conservative arithmetic ensemble mean (EM) from the NCEP Global Wave Ensemble Forecast System (GWES) to a nonlinear mapping that better captures the differences among the ensemble members and reduces the systematic and scatter errors of the forecasts. The ANNs have the 20 members of the GWES as input, and outputs are trained using observations from six buoys. The variables selected for the study are the 10-m wind speed (U10), significant wave height (Hs), and peak period (Tp) for the year of 2016. ANNs were built with one hidden layer using a hyperbolic tangent basis function. Several architectures with 12 different combinations of neurons, eight different filtering windows (time domain), and 100 seeds for the random initialization were studied and constructed for specific forecast days from 0 to 10. The results show that a small number of neurons are sufficient to reduce the bias, while 35–50 neurons produce the greatest reduction in both the scatter and systematic errors. The main advantage of the methodology using ANNs is not on short-range forecasts but at longer forecast ranges beyond 4 days. The nonlinear ensemble averaging using ANNs was able to improve the correlation coefficient on forecast day 10 from 0.39 to 0.61 for U10, from 0.50 to 0.76 for Hs, and from 0.38 to 0.63 for Tp, representing a gain of five forecast days when compared to the EM currently implemented.
Abstract
Artificial neural networks (ANNs) applied to nonlinear wave ensemble averaging are developed and studied for Gulf of Mexico simulations. It is an approach that expands the conservative arithmetic ensemble mean (EM) from the NCEP Global Wave Ensemble Forecast System (GWES) to a nonlinear mapping that better captures the differences among the ensemble members and reduces the systematic and scatter errors of the forecasts. The ANNs have the 20 members of the GWES as input, and outputs are trained using observations from six buoys. The variables selected for the study are the 10-m wind speed (U10), significant wave height (Hs), and peak period (Tp) for the year of 2016. ANNs were built with one hidden layer using a hyperbolic tangent basis function. Several architectures with 12 different combinations of neurons, eight different filtering windows (time domain), and 100 seeds for the random initialization were studied and constructed for specific forecast days from 0 to 10. The results show that a small number of neurons are sufficient to reduce the bias, while 35–50 neurons produce the greatest reduction in both the scatter and systematic errors. The main advantage of the methodology using ANNs is not on short-range forecasts but at longer forecast ranges beyond 4 days. The nonlinear ensemble averaging using ANNs was able to improve the correlation coefficient on forecast day 10 from 0.39 to 0.61 for U10, from 0.50 to 0.76 for Hs, and from 0.38 to 0.63 for Tp, representing a gain of five forecast days when compared to the EM currently implemented.
