Browse
Abstract
Weather predictions 2–4 weeks in advance, called the subseasonal time scale, are highly relevant for socioeconomic decision-makers. Unfortunately, the skill of numerical weather prediction models at this time scale is generally low. Here, we use probabilistic random forest (RF)-based machine learning models to postprocess the subseasonal to seasonal (S2S) reforecasts of the European Centre for Medium-Range Weather Forecasts (ECMWF). We show that these models are able to improve the forecasts slightly in a 20-winter mean at lead times of 14, 21, and 28 days for wintertime central European mean 2-m temperatures compared to the lead-time-dependent mean bias-corrected ECMWF’s S2S reforecasts and RF-based models using only reanalysis data as input. Predictions of the occurrence of cold wave days are improved at lead times of 21 and 28 days. Thereby, forecasts of continuous temperatures show a better skill than forecasts of binary occurrences of cold wave days. Furthermore, we analyze if the skill depends on the large-scale flow configuration of the atmosphere at initialization, as represented by weather regimes (WRs). We find that the WR at the start of the forecast influences the skill and its evolution across lead times. These results can be used to assess the conditional improvement of forecasts initialized during one WR in comparison to forecasts initialized during another WR.
Significance Statement
Forecasts of winter temperatures and cold waves 2–4 weeks in advance done by numerical weather prediction (NWP) models are often unsatisfactory due to the chaotic characteristics of the atmosphere and limited predictive skill at this time range. Here, we use statistical methods, belonging to the so-called machine learning (ML) models, to improve forecast quality by postprocessing predictions of a state-of-the-art NWP model. We compare the forecasts of the NWP and ML models considering different weather regimes (WRs), which represent the large-scale atmospheric circulation such as the typical westerly winds in Europe. We find that the ML models generally yield better temperature forecasts for 14, 21, and 28 days in advance and better forecasts of cold wave days 21 and 28 days in advance. The quality of forecasts depends on the WR present at the forecast start. This information can be used to assess the conditional improvement of forecasts.
Abstract
Weather predictions 2–4 weeks in advance, called the subseasonal time scale, are highly relevant for socioeconomic decision-makers. Unfortunately, the skill of numerical weather prediction models at this time scale is generally low. Here, we use probabilistic random forest (RF)-based machine learning models to postprocess the subseasonal to seasonal (S2S) reforecasts of the European Centre for Medium-Range Weather Forecasts (ECMWF). We show that these models are able to improve the forecasts slightly in a 20-winter mean at lead times of 14, 21, and 28 days for wintertime central European mean 2-m temperatures compared to the lead-time-dependent mean bias-corrected ECMWF’s S2S reforecasts and RF-based models using only reanalysis data as input. Predictions of the occurrence of cold wave days are improved at lead times of 21 and 28 days. Thereby, forecasts of continuous temperatures show a better skill than forecasts of binary occurrences of cold wave days. Furthermore, we analyze if the skill depends on the large-scale flow configuration of the atmosphere at initialization, as represented by weather regimes (WRs). We find that the WR at the start of the forecast influences the skill and its evolution across lead times. These results can be used to assess the conditional improvement of forecasts initialized during one WR in comparison to forecasts initialized during another WR.
Significance Statement
Forecasts of winter temperatures and cold waves 2–4 weeks in advance done by numerical weather prediction (NWP) models are often unsatisfactory due to the chaotic characteristics of the atmosphere and limited predictive skill at this time range. Here, we use statistical methods, belonging to the so-called machine learning (ML) models, to improve forecast quality by postprocessing predictions of a state-of-the-art NWP model. We compare the forecasts of the NWP and ML models considering different weather regimes (WRs), which represent the large-scale atmospheric circulation such as the typical westerly winds in Europe. We find that the ML models generally yield better temperature forecasts for 14, 21, and 28 days in advance and better forecasts of cold wave days 21 and 28 days in advance. The quality of forecasts depends on the WR present at the forecast start. This information can be used to assess the conditional improvement of forecasts.
Abstract
Long-term environmental monitoring is critical for managing the soil and groundwater at contaminated sites. Recent improvements in state-of-the-art sensor technology, communication networks, and artificial intelligence have created opportunities to modernize this monitoring activity for automated, fast, robust, and predictive monitoring. In such modernization, it is required that sensor locations be optimized to capture the spatiotemporal dynamics of all monitoring variables as well as to make it cost-effective. The legacy monitoring datasets of the target area are important to perform this optimization. In this study, we have developed a machine-learning approach to optimize sensor locations for soil and groundwater monitoring based on ensemble supervised learning and majority voting. For spatial optimization, Gaussian process regression (GPR) is used for spatial interpolation, while the majority voting is applied to accommodate the multivariate temporal dimension. Results show that the algorithms significantly outperform the random selection of the sensor locations for predictive spatiotemporal interpolation. While the method has been applied to a four-dimensional dataset (with two-dimensional space, time, and multiple contaminants), we anticipate that it can be generalizable to higher-dimensional datasets for environmental monitoring sensor location optimization.
Abstract
Long-term environmental monitoring is critical for managing the soil and groundwater at contaminated sites. Recent improvements in state-of-the-art sensor technology, communication networks, and artificial intelligence have created opportunities to modernize this monitoring activity for automated, fast, robust, and predictive monitoring. In such modernization, it is required that sensor locations be optimized to capture the spatiotemporal dynamics of all monitoring variables as well as to make it cost-effective. The legacy monitoring datasets of the target area are important to perform this optimization. In this study, we have developed a machine-learning approach to optimize sensor locations for soil and groundwater monitoring based on ensemble supervised learning and majority voting. For spatial optimization, Gaussian process regression (GPR) is used for spatial interpolation, while the majority voting is applied to accommodate the multivariate temporal dimension. Results show that the algorithms significantly outperform the random selection of the sensor locations for predictive spatiotemporal interpolation. While the method has been applied to a four-dimensional dataset (with two-dimensional space, time, and multiple contaminants), we anticipate that it can be generalizable to higher-dimensional datasets for environmental monitoring sensor location optimization.
Abstract
We introduce a machine learned surrogate model from high-resolution simulation data to capture the subgrid-scale effects in dry, stratified atmospheric flows. We use deep neural networks (NNs) to model the spatially local state differences between a coarse-resolution simulation and a high-resolution simulation. The setup enables the capture of both dissipative and antidissipative effects in the state differences. The NN model is able to accurately capture the state differences in offline tests outside the training regime. In online tests intended for production use, the NN-coupled coarse simulation has higher accuracy over a significant period of time compared to the coarse-resolution simulation without any correction. We provide evidence of the capability of the NN model to accurately capture high-gradient regions in the flow field. With the accumulation of the errors, the NN-coupled simulation becomes computationally unstable after approximately 90 coarse simulation time steps. Insights gained from these surrogate models further pave the way for formulating stable, complex, physics-based spatially local NN models which are driven by traditional subgrid-scale turbulence closure models.
Significance Statement
Flows in the atmosphere are highly chaotic and turbulent, comprising flow structures of broad scales. For effective computational modeling of atmospheric flows, the effects of the small- and large-scale structures need to be captured by the simulations. Capturing the small-scale structures requires fine-resolution simulations. Even with the current state-of-the-art supercomputers, it can be prohibitively expensive to simulate these flows when computed for the entire earth over climate time scales. Thus, it is necessary to focus on the larger-scale structures using a coarse-resolution simulation while capturing the effects of the smaller-scale structures using some parameterization (approximation) scheme and incorporating it into the coarse-resolution simulation. We use machine learning to model the effects of the small-scale structures (subgrid-scale effects) in atmospheric flows. Data from a fine-resolution simulation is used to compute the missing subgrid-scale effects in coarse-resolution simulations. We then use machine learning models to approximate these differences between the coarse- and fine-resolution simulations. We see improved accuracy for the coarse-resolution simulations when corrected using these machine learned models.
Abstract
We introduce a machine learned surrogate model from high-resolution simulation data to capture the subgrid-scale effects in dry, stratified atmospheric flows. We use deep neural networks (NNs) to model the spatially local state differences between a coarse-resolution simulation and a high-resolution simulation. The setup enables the capture of both dissipative and antidissipative effects in the state differences. The NN model is able to accurately capture the state differences in offline tests outside the training regime. In online tests intended for production use, the NN-coupled coarse simulation has higher accuracy over a significant period of time compared to the coarse-resolution simulation without any correction. We provide evidence of the capability of the NN model to accurately capture high-gradient regions in the flow field. With the accumulation of the errors, the NN-coupled simulation becomes computationally unstable after approximately 90 coarse simulation time steps. Insights gained from these surrogate models further pave the way for formulating stable, complex, physics-based spatially local NN models which are driven by traditional subgrid-scale turbulence closure models.
Significance Statement
Flows in the atmosphere are highly chaotic and turbulent, comprising flow structures of broad scales. For effective computational modeling of atmospheric flows, the effects of the small- and large-scale structures need to be captured by the simulations. Capturing the small-scale structures requires fine-resolution simulations. Even with the current state-of-the-art supercomputers, it can be prohibitively expensive to simulate these flows when computed for the entire earth over climate time scales. Thus, it is necessary to focus on the larger-scale structures using a coarse-resolution simulation while capturing the effects of the smaller-scale structures using some parameterization (approximation) scheme and incorporating it into the coarse-resolution simulation. We use machine learning to model the effects of the small-scale structures (subgrid-scale effects) in atmospheric flows. Data from a fine-resolution simulation is used to compute the missing subgrid-scale effects in coarse-resolution simulations. We then use machine learning models to approximate these differences between the coarse- and fine-resolution simulations. We see improved accuracy for the coarse-resolution simulations when corrected using these machine learned models.
Abstract
Regional climate models (RCMs) are essential tools for simulating and studying regional climate variability and change. However, their high computational cost limits the production of comprehensive ensembles of regional climate projections covering multiple scenarios and driving Global climate models (GCMs) across regions. RCM emulators based on deep learning models have recently been introduced as a cost-effective and promising alternative that requires only short RCM simulations to train the models. Therefore, evaluating their transferability to different periods, scenarios, and GCMs becomes a pivotal and complex task in which the inherent biases of both GCMs and RCMs play a significant role. Here, we focus on this problem by considering the two different emulation approaches introduced in the literature as perfect and imperfect, that we here refer to as perfect prognosis (PP) and model output statistics (MOS), respectively, following the well-established downscaling terminology. In addition to standard evaluation techniques, we expand the analysis with methods from the field of explainable artificial intelligence (XAI), to assess the physical consistency of the empirical links learnt by the models. We find that both approaches are able to emulate certain climatological properties of RCMs for different periods and scenarios (soft transferability), but the consistency of the emulation functions differs between approaches. Whereas PP learns robust and physically meaningful patterns, MOS results are GCM dependent and lack physical consistency in some cases. Both approaches face problems when transferring the emulation function to other GCMs (hard transferability), due to the existence of GCM-dependent biases. This limits their applicability to build RCM ensembles. We conclude by giving prospects for future applications.
Significance Statement
Regional climate model (RCM) emulators are a cost-effective emerging approach for generating comprehensive ensembles of regional climate projections. Promising results have been recently obtained using deep learning models. However, their potential to capture the regional climate dynamics and to emulate other periods, emission scenarios, or driving global climate models (GCMs) remains an open issue that affects their practical use. This study explores the potential of current emulation approaches incorporating new explainable artificial intelligence (XAI) evaluation techniques to assess the reliability and transferability of the emulators. Our findings show that the different global and regional model biases involved in the different approaches play a key role in transferability. Based on the results obtained, we provide some prospects for potential applications of these models in challenging problems.
Abstract
Regional climate models (RCMs) are essential tools for simulating and studying regional climate variability and change. However, their high computational cost limits the production of comprehensive ensembles of regional climate projections covering multiple scenarios and driving Global climate models (GCMs) across regions. RCM emulators based on deep learning models have recently been introduced as a cost-effective and promising alternative that requires only short RCM simulations to train the models. Therefore, evaluating their transferability to different periods, scenarios, and GCMs becomes a pivotal and complex task in which the inherent biases of both GCMs and RCMs play a significant role. Here, we focus on this problem by considering the two different emulation approaches introduced in the literature as perfect and imperfect, that we here refer to as perfect prognosis (PP) and model output statistics (MOS), respectively, following the well-established downscaling terminology. In addition to standard evaluation techniques, we expand the analysis with methods from the field of explainable artificial intelligence (XAI), to assess the physical consistency of the empirical links learnt by the models. We find that both approaches are able to emulate certain climatological properties of RCMs for different periods and scenarios (soft transferability), but the consistency of the emulation functions differs between approaches. Whereas PP learns robust and physically meaningful patterns, MOS results are GCM dependent and lack physical consistency in some cases. Both approaches face problems when transferring the emulation function to other GCMs (hard transferability), due to the existence of GCM-dependent biases. This limits their applicability to build RCM ensembles. We conclude by giving prospects for future applications.
Significance Statement
Regional climate model (RCM) emulators are a cost-effective emerging approach for generating comprehensive ensembles of regional climate projections. Promising results have been recently obtained using deep learning models. However, their potential to capture the regional climate dynamics and to emulate other periods, emission scenarios, or driving global climate models (GCMs) remains an open issue that affects their practical use. This study explores the potential of current emulation approaches incorporating new explainable artificial intelligence (XAI) evaluation techniques to assess the reliability and transferability of the emulators. Our findings show that the different global and regional model biases involved in the different approaches play a key role in transferability. Based on the results obtained, we provide some prospects for potential applications of these models in challenging problems.
Abstract
Producing high-quality forecasts of key climate variables, such as temperature and precipitation, on subseasonal time scales has long been a gap in operational forecasting. This study explores an application of machine learning (ML) models as postprocessing tools for subseasonal forecasting. Lagged numerical ensemble forecasts (i.e., an ensemble where the members have different initialization dates) and observational data, including relative humidity, pressure at sea level, and geopotential height, are incorporated into various ML methods to predict monthly average precipitation and 2-m temperature 2 weeks in advance for the continental United States. For regression, quantile regression, and tercile classification tasks, we consider using linear models, random forests, convolutional neural networks, and stacked models (a multimodel approach based on the prediction of the individual ML models). Unlike previous ML approaches that often use ensemble mean alone, we leverage information embedded in the ensemble forecasts to enhance prediction accuracy. Additionally, we investigate extreme event predictions that are crucial for planning and mitigation efforts. Considering ensemble members as a collection of spatial forecasts, we explore different approaches to using spatial information. Trade-offs between different approaches may be mitigated with model stacking. Our proposed models outperform standard baselines such as climatological forecasts and ensemble means. In addition, we investigate feature importance, trade-offs between using the full ensemble or only the ensemble mean, and different modes of accounting for spatial variability.
Significance Statement
Accurately forecasting temperature and precipitation on subseasonal time scales—2 weeks–2 months in advance—is extremely challenging. These forecasts would have immense value in agriculture, insurance, and economics. Our paper describes an application of machine learning techniques to improve forecasts of monthly average precipitation and 2-m temperature using lagged physics-based predictions and observational data 2 weeks in advance for the entire continental United States. For lagged ensembles, the proposed models outperform standard benchmarks such as historical averages and averages of physics-based predictions. Our findings suggest that utilizing the full set of physics-based predictions instead of the average enhances the accuracy of the final forecast.
Abstract
Producing high-quality forecasts of key climate variables, such as temperature and precipitation, on subseasonal time scales has long been a gap in operational forecasting. This study explores an application of machine learning (ML) models as postprocessing tools for subseasonal forecasting. Lagged numerical ensemble forecasts (i.e., an ensemble where the members have different initialization dates) and observational data, including relative humidity, pressure at sea level, and geopotential height, are incorporated into various ML methods to predict monthly average precipitation and 2-m temperature 2 weeks in advance for the continental United States. For regression, quantile regression, and tercile classification tasks, we consider using linear models, random forests, convolutional neural networks, and stacked models (a multimodel approach based on the prediction of the individual ML models). Unlike previous ML approaches that often use ensemble mean alone, we leverage information embedded in the ensemble forecasts to enhance prediction accuracy. Additionally, we investigate extreme event predictions that are crucial for planning and mitigation efforts. Considering ensemble members as a collection of spatial forecasts, we explore different approaches to using spatial information. Trade-offs between different approaches may be mitigated with model stacking. Our proposed models outperform standard baselines such as climatological forecasts and ensemble means. In addition, we investigate feature importance, trade-offs between using the full ensemble or only the ensemble mean, and different modes of accounting for spatial variability.
Significance Statement
Accurately forecasting temperature and precipitation on subseasonal time scales—2 weeks–2 months in advance—is extremely challenging. These forecasts would have immense value in agriculture, insurance, and economics. Our paper describes an application of machine learning techniques to improve forecasts of monthly average precipitation and 2-m temperature using lagged physics-based predictions and observational data 2 weeks in advance for the entire continental United States. For lagged ensembles, the proposed models outperform standard benchmarks such as historical averages and averages of physics-based predictions. Our findings suggest that utilizing the full set of physics-based predictions instead of the average enhances the accuracy of the final forecast.
Abstract
Combining strengths from deep learning and extreme value theory can help describe complex relationships between variables where extreme events have significant impacts (e.g., environmental or financial applications). Neural networks learn complicated nonlinear relationships from large datasets under limited parametric assumptions. By definition, the number of occurrences of extreme events is small, which limits the ability of the data-hungry, nonparametric neural network to describe rare events. Inspired by recent extreme cold winter weather events in North America caused by atmospheric blocking, we examine several probabilistic generative models for the entire multivariate probability distribution of daily boreal winter surface air temperature. We propose metrics to measure spatial asymmetries, such as long-range anticorrelated patterns that commonly appear in temperature fields during blocking events. Compared to vine copulas, the statistical standard for multivariate copula modeling, deep learning methods show improved ability to reproduce complicated asymmetries in the spatial distribution of ERA5 temperature reanalysis, including the spatial extent of in-sample extreme events.
Abstract
Combining strengths from deep learning and extreme value theory can help describe complex relationships between variables where extreme events have significant impacts (e.g., environmental or financial applications). Neural networks learn complicated nonlinear relationships from large datasets under limited parametric assumptions. By definition, the number of occurrences of extreme events is small, which limits the ability of the data-hungry, nonparametric neural network to describe rare events. Inspired by recent extreme cold winter weather events in North America caused by atmospheric blocking, we examine several probabilistic generative models for the entire multivariate probability distribution of daily boreal winter surface air temperature. We propose metrics to measure spatial asymmetries, such as long-range anticorrelated patterns that commonly appear in temperature fields during blocking events. Compared to vine copulas, the statistical standard for multivariate copula modeling, deep learning methods show improved ability to reproduce complicated asymmetries in the spatial distribution of ERA5 temperature reanalysis, including the spatial extent of in-sample extreme events.
Abstract
We use a spectral-based approach that employs principal component analysis along with a relatively shallow artificial neural network (NN) to substantially reduce noise and other artifacts in terrestrial chlorophyll solar-induced fluorescence (SIF) retrievals. SIF is a very small emission at red and far-red wavelengths that is difficult to measure and is highly sensitive to random errors and systematic artifacts. Our approach relies upon an assumption that a trained NN can effectively reconstruct the total SIF signal from a relatively small number of leading principal components of the satellite-observed far-red radiance spectra without using information from the trailing modes that contain most of the random errors. We test the approach with simulated reflectance spectra produced with a full atmospheric and surface radiative transfer model using different observing and geophysical parameters and various noise levels. The resulting noisy and noise-reduced retrieved SIF values are compared with true values to assess performance. We then apply our noise reduction approach to SIF derived from two different satellite spectrometers. For evaluation, since the truth in this case is unknown, we compare SIF retrievals from two independent sensors with each other. We also compare the noise-reduced SIF temporal variations with those from an independent gross primary product (GPP) product that should display similar variations. Results show that our noise reduction approach improves the capture of SIF seasonal and interannual variability. Our approach should be applicable to many noisy data products derived from spectral measurements. Our methodology does not replace the original retrieval algorithms; rather, the original noisy retrievals are needed as the target for the NN training process.
Significance Statement
The purpose of this study is to document and demonstrate a machine learning algorithm that is used to effectively reduce noise and artifacts in a satellite data product, solar-induced fluorescence (SIF) from chlorophyll. This is important because SIF retrievals are typically noisy, and the noise limits their ability to be used for diagnosing plant health and productivity. Our results show substantial improvement in SIF retrievals that may lead to new applications. Our approach can be similarly applied to other noisy satellite data products.
Abstract
We use a spectral-based approach that employs principal component analysis along with a relatively shallow artificial neural network (NN) to substantially reduce noise and other artifacts in terrestrial chlorophyll solar-induced fluorescence (SIF) retrievals. SIF is a very small emission at red and far-red wavelengths that is difficult to measure and is highly sensitive to random errors and systematic artifacts. Our approach relies upon an assumption that a trained NN can effectively reconstruct the total SIF signal from a relatively small number of leading principal components of the satellite-observed far-red radiance spectra without using information from the trailing modes that contain most of the random errors. We test the approach with simulated reflectance spectra produced with a full atmospheric and surface radiative transfer model using different observing and geophysical parameters and various noise levels. The resulting noisy and noise-reduced retrieved SIF values are compared with true values to assess performance. We then apply our noise reduction approach to SIF derived from two different satellite spectrometers. For evaluation, since the truth in this case is unknown, we compare SIF retrievals from two independent sensors with each other. We also compare the noise-reduced SIF temporal variations with those from an independent gross primary product (GPP) product that should display similar variations. Results show that our noise reduction approach improves the capture of SIF seasonal and interannual variability. Our approach should be applicable to many noisy data products derived from spectral measurements. Our methodology does not replace the original retrieval algorithms; rather, the original noisy retrievals are needed as the target for the NN training process.
Significance Statement
The purpose of this study is to document and demonstrate a machine learning algorithm that is used to effectively reduce noise and artifacts in a satellite data product, solar-induced fluorescence (SIF) from chlorophyll. This is important because SIF retrievals are typically noisy, and the noise limits their ability to be used for diagnosing plant health and productivity. Our results show substantial improvement in SIF retrievals that may lead to new applications. Our approach can be similarly applied to other noisy satellite data products.
Abstract
The safe and efficient execution of offshore operations requires short-term (1–6 h ahead) high-quality probabilistic forecasts of metocean variables. The development areas for offshore wind projects, potentially in high depths, make it difficult to gather measurement data. This paper explores the use of deep learning for wind speed forecasting at an unobserved offshore location. The proposed convolutional architecture jointly exploits coastal measurements and numerical weather predictions to emulate multivariate probabilistic short-term forecasts. We explore both Gaussian and non-Gaussian neural representations using normalizing flows. We benchmark these approaches with respect to state-of-the-art data-driven schemes, including analog methods and quantile forecasting. The performance of the models and resulting forecast quality are analyzed in terms of probabilistic calibration, probabilistic and deterministic metrics, and as a function of weather situations. We report numerical experiments for a real case study off the French Mediterranean coast. Our results highlight the role of regional numerical weather prediction and coastal in situ measurement in the performance of postprocessing. For single-valued forecasts, a 40% decrease in RMSE is observed compared to the direct use of numerical weather predictions. Significant skill improvements are also obtained for the probabilistic forecasts, in terms of various scores, as well as an acceptable probabilistic calibration. The proposed architecture can process a large amount of heterogeneous input data and offers a versatile probabilistic framework for multivariate forecasting.
Abstract
The safe and efficient execution of offshore operations requires short-term (1–6 h ahead) high-quality probabilistic forecasts of metocean variables. The development areas for offshore wind projects, potentially in high depths, make it difficult to gather measurement data. This paper explores the use of deep learning for wind speed forecasting at an unobserved offshore location. The proposed convolutional architecture jointly exploits coastal measurements and numerical weather predictions to emulate multivariate probabilistic short-term forecasts. We explore both Gaussian and non-Gaussian neural representations using normalizing flows. We benchmark these approaches with respect to state-of-the-art data-driven schemes, including analog methods and quantile forecasting. The performance of the models and resulting forecast quality are analyzed in terms of probabilistic calibration, probabilistic and deterministic metrics, and as a function of weather situations. We report numerical experiments for a real case study off the French Mediterranean coast. Our results highlight the role of regional numerical weather prediction and coastal in situ measurement in the performance of postprocessing. For single-valued forecasts, a 40% decrease in RMSE is observed compared to the direct use of numerical weather predictions. Significant skill improvements are also obtained for the probabilistic forecasts, in terms of various scores, as well as an acceptable probabilistic calibration. The proposed architecture can process a large amount of heterogeneous input data and offers a versatile probabilistic framework for multivariate forecasting.
Abstract
Machine learning (ML) has rapidly transitioned from a niche activity to a mainstream tool for environmental research applications including atmospheric science cloud microphysics studies. Two recently developed cloud particle probes measure the light scattered in the near forward direction and save digital images of the scattering light. Scattering pattern images collected by the Particle Phase Discriminator mark 2, Karlsruhe edition (PPD-2K), and the Small Ice Detector, version 3 (SID-3) provide valuable information for particle shape and size characterization. Since different particle shapes have distinctly different light scattering characteristics, the images are ideally suited for ML. Here, results of a ML project to characterize ice particle shapes sampled by the PPD-2K in ice fog and diamond dust during a 3-yr project in Fairbanks, Alaska. About 2.15 million light-scattering pattern images were collected during 3 years of measurements with the PPD-2K. Visual Geometry Group (VGG) convolutional neural network (CNN) was trained to categorize light-scattering patterns into eight categories. Initial training images (120 each category) were selected by human visual examination of data, and the training dataset was augmented using an automated iterative method for image identification of further images which were all visually inspected by a human. Results were well correlated to similar categories identified from previously developed classification algorithms. ML identifies characteristics not included in automated analysis such as sublimation. Of the 2.15 million images analyzed, 1.3% were categorized as spherical (liquid), 43.5% were categorized as having rough surfaces, 15.3% were pristine, 16.3% were categorized as sublimating, and the remaining 23.6% did not fit into any of those categories (irregular or saturated).
Significance Statement
The shapes and sizes of cloud particles can be extremely important for understanding the conditions that exist in the cloud. In this study, we show that more information about cloud particle characteristics can be identified by using machine learning than by traditional means. To demonstrate this, data are analyzed from a 3-yr study of ice fog and diamond dust events in Fairbanks, Alaska. The Particle Phase Discriminator instrument collected 2.15 million light-scattering pattern images of cloud particles during ground-based measurements. Neither traditional techniques nor machine learning were able to identify all categories, but a combination of both techniques led to a more complete view of ice particle shapes.
Abstract
Machine learning (ML) has rapidly transitioned from a niche activity to a mainstream tool for environmental research applications including atmospheric science cloud microphysics studies. Two recently developed cloud particle probes measure the light scattered in the near forward direction and save digital images of the scattering light. Scattering pattern images collected by the Particle Phase Discriminator mark 2, Karlsruhe edition (PPD-2K), and the Small Ice Detector, version 3 (SID-3) provide valuable information for particle shape and size characterization. Since different particle shapes have distinctly different light scattering characteristics, the images are ideally suited for ML. Here, results of a ML project to characterize ice particle shapes sampled by the PPD-2K in ice fog and diamond dust during a 3-yr project in Fairbanks, Alaska. About 2.15 million light-scattering pattern images were collected during 3 years of measurements with the PPD-2K. Visual Geometry Group (VGG) convolutional neural network (CNN) was trained to categorize light-scattering patterns into eight categories. Initial training images (120 each category) were selected by human visual examination of data, and the training dataset was augmented using an automated iterative method for image identification of further images which were all visually inspected by a human. Results were well correlated to similar categories identified from previously developed classification algorithms. ML identifies characteristics not included in automated analysis such as sublimation. Of the 2.15 million images analyzed, 1.3% were categorized as spherical (liquid), 43.5% were categorized as having rough surfaces, 15.3% were pristine, 16.3% were categorized as sublimating, and the remaining 23.6% did not fit into any of those categories (irregular or saturated).
Significance Statement
The shapes and sizes of cloud particles can be extremely important for understanding the conditions that exist in the cloud. In this study, we show that more information about cloud particle characteristics can be identified by using machine learning than by traditional means. To demonstrate this, data are analyzed from a 3-yr study of ice fog and diamond dust events in Fairbanks, Alaska. The Particle Phase Discriminator instrument collected 2.15 million light-scattering pattern images of cloud particles during ground-based measurements. Neither traditional techniques nor machine learning were able to identify all categories, but a combination of both techniques led to a more complete view of ice particle shapes.
Abstract
Convective initiation (CI) nowcasting remains a challenging problem for both numerical weather prediction models and existing nowcasting algorithms. In this study, an object-based probabilistic deep learning model is developed to predict CI based on multichannel infrared GOES-16 satellite observations. The data come from patches surrounding potential CI events identified in Multi-Radar Multi-Sensor Doppler weather radar products over the Great Plains region from June and July 2020 and June 2021. An objective radar-based approach is used to identify these events. The deep learning model significantly outperforms the classical logistic model at lead times up to 1 h, especially on the false alarm ratio. Through case studies, the deep learning model exhibits dependence on the characteristics of clouds and moisture at multiple altitudes. Model explanation further reveals that the contribution of features to model predictions is significantly dependent on the baseline, a reference point against which the prediction is compared. Under a moist baseline, moisture gradients in the lower and middle troposphere contribute most to correct CI forecasts. In contrast, under a clear-sky baseline, correct CI forecasts are dominated by cloud-top features, including cloud-top glaciation, height, and cloud coverage. Our study demonstrates the advantage of using different baselines in further understanding model behavior and gaining scientific insights.
Abstract
Convective initiation (CI) nowcasting remains a challenging problem for both numerical weather prediction models and existing nowcasting algorithms. In this study, an object-based probabilistic deep learning model is developed to predict CI based on multichannel infrared GOES-16 satellite observations. The data come from patches surrounding potential CI events identified in Multi-Radar Multi-Sensor Doppler weather radar products over the Great Plains region from June and July 2020 and June 2021. An objective radar-based approach is used to identify these events. The deep learning model significantly outperforms the classical logistic model at lead times up to 1 h, especially on the false alarm ratio. Through case studies, the deep learning model exhibits dependence on the characteristics of clouds and moisture at multiple altitudes. Model explanation further reveals that the contribution of features to model predictions is significantly dependent on the baseline, a reference point against which the prediction is compared. Under a moist baseline, moisture gradients in the lower and middle troposphere contribute most to correct CI forecasts. In contrast, under a clear-sky baseline, correct CI forecasts are dominated by cloud-top features, including cloud-top glaciation, height, and cloud coverage. Our study demonstrates the advantage of using different baselines in further understanding model behavior and gaining scientific insights.
