Browse

You are looking at 1 - 10 of 20 items for :

  • The 1st NOAA Workshop on Leveraging AI in the Exploitation of Satellite Earth Observations & Numerical Weather Prediction x
  • Refine by Access: Content accessible to me x
Clear All
Maria J. Molina
,
Travis A. O’Brien
,
Gemma Anderson
,
Moetasim Ashfaq
,
Katrina E. Bennett
,
William D. Collins
,
Katherine Dagon
,
Juan M. Restrepo
, and
Paul A. Ullrich

Abstract

Climate variability and weather phenomena can cause extremes and pose significant risk to society and ecosystems, making continued advances in our physical understanding of such events of utmost importance for regional and global security. Advances in machine learning (ML) have been leveraged for applications in climate variability and weather, empowering scientists to approach questions using big data in new ways. Growing interest across the scientific community in these areas has motivated coordination between the physical and computer science disciplines to further advance the state of the science and tackle pressing challenges. During a recently held workshop that had participants across academia, private industry, and research laboratories, it became clear that a comprehensive review of recent and emerging ML applications for climate variability and weather phenomena that can cause extremes was needed. This article aims to fulfill this need by discussing recent advances, challenges, and research priorities in the following topics: sources of predictability for modes of climate variability, feature detection, extreme weather and climate prediction and precursors, observation–model integration, downscaling, and bias correction. This article provides a review for domain scientists seeking to incorporate ML into their research. It also provides a review for those with some ML experience seeking to broaden their knowledge of ML applications for climate variability and weather.

Open access
Yun Fan
,
Vladimir Krasnopolsky
,
Huug van den Dool
,
Chung-Yu Wu
, and
Jon Gottschalck

Abstract

Forecast skill from dynamical forecast models decreases quickly with projection time due to various errors. Therefore, postprocessing methods, from simple bias correction methods to more complicated multiple linear regression–based model output statistics, are used to improve raw model forecasts. Usually, these methods show clear forecast improvement over the raw model forecasts, especially for short-range weather forecasts. However, linear approaches have limitations because the relationship between predictands and predictors may be nonlinear. This is even truer for extended range forecasts, such as week-3–4 forecasts. In this study, neural network techniques are used to seek or model the relationships between a set of predictors and predictands, and eventually to improve week-3–4 precipitation and 2-m temperature forecasts made by the NOAA/NCEP Climate Forecast System. Benefitting from advances in machine learning techniques in recent years, more flexible and capable machine learning algorithms and availability of big datasets enable us not only to explore nonlinear features or relationships within a given large dataset, but also to extract more sophisticated pattern relationships and covariabilities hidden within the multidimensional predictors and predictands. Then these more sophisticated relationships and high-level statistical information are used to correct the model week-3–4 precipitation and 2-m temperature forecasts. The results show that to some extent neural network techniques can significantly improve the week-3–4 forecast accuracy and greatly increase the efficiency over the traditional multiple linear regression methods.

Free access
Soni Yatheendradas
and
Sujay Kumar

Abstract

Satellite-based remotely sensed observations of snow cover fraction (SCF) can have data gaps in spatially distributed coverage from sensor and orbital limitations. We mitigate these limitations in the example fine-resolution Moderate Resolution Imaging Spectroradiometer (MODIS) data by gap-filling using auxiliary 1-km datasets that either aid in downscaling from coarser-resolution (5 km) MODIS SCF wherever not fully covered by clouds, or else by themselves via regression wherever fully cloud covered. This study’s prototype predicts a 1-km version of the 500-m MOD10A1 SCF target. Due to noncollocatedness of spatial gaps even across input and auxiliary datasets, we consider a recent gap-agnostic advancement of partial convolution in computer vision for both training and predictive gap-filling. Partial convolution accommodates spatially consistent gaps across the input images, effectively implementing a two-dimensional masking. To overcome reduced usable data from noncollocated spatial gaps across inputs, we innovate a fully generalized three-dimensional masking in this partial convolution. This enables a valid output value at a pixel even if only a single valid input variable and its value exist in the neighborhood covered by the convolutional filter zone centered around that pixel. Thus, our gap-agnostic technique can use significantly more examples for training (∼67%) and prediction (∼100%), instead of only less than 10% for the previous partial convolution. We train an example simple three-layer legacy super-resolution convolutional neural network (SRCNN) to obtain downscaling and regression component performances that are better than baseline values of either climatology or MOD10C1 SCF as relevant. Our generalized partial convolution can enable multiple Earth science applications like downscaling, regression, classification, and segmentation that were hindered by data gaps.

Full access
Akhil Sanjay Potdar
,
Pierre-Emmanuel Kirstetter
,
Devon Woods
, and
Manabendra Saharia

Abstract

In the hydrological sciences, the outstanding challenge of regional modeling requires to capture common and event-specific hydrologic behaviors driven by rainfall spatial variability and catchment physiography during floods. The overall objective of this study is to develop robust understanding and predictive capability of how rainfall spatial variability influences flood peak discharge relative to basin physiography. A machine-learning approach is used on a high-resolution dataset of rainfall and flooding events spanning 10 years, with rainfall events and basins of widely varying characteristics selected across the continental United States. It overcomes major limitations in prior studies that were based on limited observations or hydrological model simulations. This study explores first-order dependencies in the relationships between peak discharge, rainfall variability, and basin physiography, and it sheds light on these complex interactions using a multidimensional statistical modeling approach. Among different machine-learning techniques, XGBoost is used to determine the significant physiographical and rainfall characteristics that influence peak discharge through variable importance analysis. A parsimonious model with low bias and variance is created that can be deployed in the future for flash flood forecasting. The results confirm that, although the spatial organization of rainfall within a basin has a major influence on basin response, basin physiography is the primary driver of peak discharge. These findings have unprecedented spatial and temporal representativeness in terms of flood characterization across basins. An improved understanding of subbasin scale rainfall spatial variability will aid in robust flash flood characterization as well as with identifying basins that could most benefit from distributed hydrologic modeling.

Full access
Dan Lu
,
Goutam Konapala
,
Scott L. Painter
,
Shih-Chieh Kao
, and
Sudershan Gangrade

Abstract

Hydrologic predictions at rural watersheds are important but also challenging due to data shortage. Long short-term memory (LSTM) networks are a promising machine learning approach and have demonstrated good performance in streamflow predictions. However, due to its data-hungry nature, most LSTM applications focus on well-monitored catchments with abundant and high-quality observations. In this work, we investigate predictive capabilities of LSTM in poorly monitored watersheds with short observation records. To address three main challenges of LSTM applications in data-scarce locations, i.e., overfitting, uncertainty quantification (UQ), and out-of-distribution prediction, we evaluate different regularization techniques to prevent overfitting, apply a Bayesian LSTM for UQ, and introduce a physics-informed hybrid LSTM to enhance out-of-distribution prediction. Through case studies in two diverse sets of catchments with and without snow influence, we demonstrate that 1) when hydrologic variability in the prediction period is similar to the calibration period, LSTM models can reasonably predict daily streamflow with Nash–Sutcliffe efficiency above 0.8, even with only 2 years of calibration data; 2) when the hydrologic variability in the prediction and calibration periods is dramatically different, LSTM alone does not predict well, but the hybrid model can improve the out-of-distribution prediction with acceptable generalization accuracy; 3) L2 norm penalty and dropout can mitigate overfitting, and Bayesian and hybrid LSTM have no overfitting; and 4) Bayesian LSTM provides useful uncertainty information to improve prediction understanding and credibility. These insights have vital implications for streamflow simulation in watersheds where data quality and availability are a critical issue.

Full access
Sid-Ahmed Boukabara
,
Vladimir Krasnopolsky
,
Stephen G. Penny
,
Jebb Q. Stewart
,
Amy McGovern
,
David Hall
,
John E. Ten Hoeve
,
Jason Hickey
,
Hung-Lung Allen Huang
,
John K. Williams
,
Kayo Ide
,
Philippe Tissot
,
Sue Ellen Haupt
,
Kenneth S. Casey
,
Nikunj Oza
,
Alan J. Geer
,
Eric S. Maddy
, and
Ross N. Hoffman

Abstract

Promising new opportunities to apply artificial intelligence (AI) to the Earth and environmental sciences are identified, informed by an overview of current efforts in the community. Community input was collected at the first National Oceanic and Atmospheric Administration (NOAA) workshop on “Leveraging AI in the Exploitation of Satellite Earth Observations and Numerical Weather Prediction” held in April 2019. This workshop brought together over 400 scientists, program managers, and leaders from the public, academic, and private sectors in order to enable experts involved in the development and adaptation of AI tools and applications to meet and exchange experiences with NOAA experts. Paths are described to actualize the potential of AI to better exploit the massive volumes of environmental data from satellite and in situ sources that are critical for numerical weather prediction (NWP) and other Earth and environmental science applications. The main lessons communicated from community input via active workshop discussions and polling are reported. Finally, recommendations are presented for both scientists and decision-makers to address some of the challenges facing the adoption of AI across all Earth science.

Open access
Yaling Liu
,
Dongdong Chen
,
Soukayna Mouatadid
,
Xiaoliang Lu
,
Min Chen
,
Yu Cheng
,
Zhenghui Xie
,
Binghao Jia
,
Huan Wu
, and
Pierre Gentine

Abstract

Soil moisture (SM) links the water and energy cycles over the land–atmosphere interface and largely determines ecosystem functionality, positioning it as an essential player in the Earth system. Despite its importance, accurate estimation of large-scale SM remains a challenge. Here we leverage the strength of neural network (NN) and fidelity of long-term measurements to develop a daily multilayer cropland SM dataset for China from 1981 to 2013, implemented for a range of different cropping patterns. The training and testing of the NN for the five soil layers (0–50 cm, 10-cm depth each) yield R 2 values of 0.65–0.70 and 0.64–0.69, respectively. Our analysis reveals that precipitation and soil properties are the two dominant factors determining SM, but cropping pattern is also crucial. In addition, our simulations of alternative cropping patterns indicate that winter wheat followed by fallow will largely alleviate the SM depletion in most parts of China. On the other hand, cropping patterns of fallow in the winter followed by maize/soybean seem to further aggravate SM decline in the Huang-Huai-Hai region and southwestern China, relative to prevalent practices of double cropping. This may be due to their low soil porosity, which results in more soil water drainage, as opposed to the case that winter crop roots help maintain SM. This multilayer cropland SM dataset with granularity of cropping patterns provides an important alternative and is complementary to modeled and satellite-retrieved products.

Full access
Andrew E. Mercer
,
Alexandria D. Grimes
, and
Kimberly M. Wood

Abstract

Tropical cyclone (TC) track forecasts have improved in recent decades while intensity forecasts, particularly predictions of rapid intensification (RI), continue to show low skill. Many statistical methods have shown promise in predicting RI using environmental fields, although these methods rely heavily upon supervised learning techniques such as classification. Advances in unsupervised learning techniques, particularly those that integrate nonlinearity into the class separation problem, can improve discrimination ability for difficult tasks such as RI prediction. This study quantifies separability between RI and non-RI environments for 2004–16 Atlantic Ocean TCs using an unsupervised learning method that blends principal component analysis with k-means cluster analysis. Input fields consisted of TC-centered 1° Global Forecast System analysis (GFSA) grids (170 different variables and isobaric levels) for 3605 TC samples and five domain sizes. Results are directly compared with separability offered by operational RI forecast predictors for eight RI definitions. The unsupervised learning procedure produced improved separability over operational predictors for all eight RI definitions, five of which showed statistically significant improvement. Composites from these best-separating GFSA fields highlighted the importance of mid- and upper-level relative humidity in identifying the onset of short-term RI, whereas long-term, higher-magnitude RI was generally associated with weaker absolute vorticity. Other useful predictors included optimal thermodynamic RI ingredients along the mean trajectory of the TC. The results suggest that the orientation of a more favorable thermodynamic environment relative to the TC and midlevel vorticity magnitudes could be useful predictors for RI.

Full access
Kyle A. Hilburn
,
Imme Ebert-Uphoff
, and
Steven D. Miller

Abstract

The objective of this research is to develop techniques for assimilating GOES-R series observations in precipitating scenes for the purpose of improving short-term convective-scale forecasts of high-impact weather hazards. Whereas one approach is radiance assimilation, the information content of GOES-R radiances from its Advanced Baseline Imager saturates in precipitating scenes, and radiance assimilation does not make use of lightning observations from the GOES Lightning Mapper. Here, a convolutional neural network (CNN) is developed to transform GOES-R radiances and lightning into synthetic radar reflectivity fields to make use of existing radar assimilation techniques. We find that the ability of CNNs to utilize spatial context is essential for this application and offers breakthrough improvement in skill compared to traditional pixel-by-pixel based approaches. To understand the improved performance, we use a novel analysis method that combines several techniques, each providing different insights into the network’s reasoning. Channel-withholding experiments and spatial information–withholding experiments are used to show that the CNN achieves skill at high reflectivity values from the information content in radiance gradients and the presence of lightning. The attribution method, layerwise relevance propagation, demonstrates that the CNN uses radiance and lightning information synergistically, where lightning helps the CNN focus on which neighboring locations are most important. Synthetic inputs are used to quantify the sensitivity to radiance gradients, showing that sharper gradients produce a stronger response in predicted reflectivity. Lightning observations are found to be uniquely valuable for their ability to pinpoint locations of strong radar echoes.

Open access
John L. Cintineo
,
Michael J. Pavolonis
,
Justin M. Sieglaff
,
Anthony Wimmers
,
Jason Brunner
, and
Willard Bellon

Abstract

Intense thunderstorms threaten life and property, impact aviation, and are a challenging forecast problem, particularly without precipitation-sensing radar data. Trained forecasters often look for features in geostationary satellite images such as rapid cloud growth, strong and persistent overshooting tops, U- or V-shaped patterns in storm-top temperature (and associated above-anvil cirrus plumes), thermal couplets, intricate texturing in cloud albedo (e.g., “bubbling” cloud tops), cloud-top divergence, spatial and temporal trends in lightning, and other nuances to identify intense thunderstorms. In this paper, a machine-learning algorithm was employed to automatically learn and extract salient features and patterns in geostationary satellite data for the prediction of intense convection. Namely, a convolutional neural network (CNN) was trained on 0.64-μm reflectance and 10.35-μm brightness temperature from the Advanced Baseline Imager (ABI) and flash-extent density (FED) from the Geostationary Lightning Mapper (GLM) on board GOES-16. Using a training dataset consisting of over 220 000 human-labeled satellite images, the CNN learned pertinent features that are known to be associated with intense convection and skillfully discriminated between intense and ordinary convection. The CNN also learned a more nuanced feature associated with intense convection—strong infrared brightness temperature gradients near cloud edges in the vicinity of the main updraft. A successive-permutation test ranked the most important predictors as follows: 1) ABI 10.35-μm brightness temperature, 2) ABI GLM flash-extent density, and 3) ABI 0.64-μm reflectance. The CNN model can provide forecasters with quantitative information that often foreshadows the occurrence of severe weather, day or night, over the full range of instrument-scan modes.

Full access