Search Results

You are looking at 1 - 7 of 7 items for

  • Author or Editor: Kyle Hilburn x
  • Refine by Access: All Content x
Clear All Modify Search
Kyle A. Hilburn

Abstract

Convolutional neural networks (CNNs) are opening new possibilities in the realm of satellite remote sensing. CNNs are especially useful for capturing the information in spatial patterns that is evident to the human eye but has eluded classical pixelwise retrieval algorithms. However, the black-box nature of CNN predictions makes them difficult to interpret, hindering their trustworthiness. This paper explores a new way to simplify CNNs that allows them to be implemented in a fully transparent and interpretable framework. This clarity is accomplished by moving the inner workings of the CNN out into a feature engineering step and replacing the CNN with a regression model. The specific example of the GOES Radar Estimation via Machine Learning to Inform NWP (GREMLIN) is used to demonstrate that such simplifications are possible and to show the benefits of the interpretable approach. GREMLIN translates images of GOES radiances and lightning into images of radar reflectivity, and previous research used explainable artificial intelligence (XAI) approaches to explain some aspects of how GREMLIN makes predictions. However, the Interpretable GREMLIN model shows that XAI missed several strategies, and XAI does not provide guarantees on how the model will respond when confronted with new scenarios. In contrast, the interpretable model establishes well-defined relationships between inputs and outputs, offering a clear mapping of the spatial context utilized by the CNN to make accurate predictions, and providing guarantees on how the model will respond to new inputs. The significance of this work is that it provides a new approach for developing trustworthy artificial intelligence models.

Significance Statement

Convolutional neural networks (CNNs) are very powerful tools for interpreting and processing satellite imagery. However, the black-box nature of their predictions makes them difficult to interpret, compromising their trustworthiness when applied in the context of high-stakes decision-making. This paper develops an interpretable version of a CNN model, showing that it has similar performance as the original CNN. The interpretable model is analyzed to obtain clear relationships between inputs and outputs, which elucidates the nature of spatial context utilized by CNNs to make accurate predictions. The interpretable model has a well-defined response to inputs, providing guarantees for how it will respond to novel inputs. The significance of this work is that it provides an approach to developing trustworthy artificial intelligence models.

Open access
Yoonjin Lee
and
Kyle Hilburn

Abstract

Geostationary Operational Environmental Satellites (GOES) Radar Estimation via Machine Learning to Inform NWP (GREMLIN) is a machine learning model that outputs composite reflectivity using GOES-R Series Advanced Baseline Imager (ABI) and Geostationary Lightning Mapper (GLM) input data. GREMLIN is useful for observing severe weather and initializing convection for short-term forecasts, especially over regions without ground-based radars. This study expands the evaluation of GREMLIN’s accuracy against the Multi-Radar Multi-Sensor (MRMS) System to the entire contiguous United States (CONUS) for the entire annual cycle. Regional and temporal variation of validation metrics are examined over CONUS by season, day of year, and time of day. Since GREMLIN was trained with data in spring and summer, root-mean-square difference (RMSD) and bias are lowest in the order of summer, spring, fall, and winter. In summer, diurnal patterns of RMSD follow those of precipitation occurrence. Winter has the highest RMSD because of cold surfaces mistaken as precipitating clouds, but some of these errors can be removed by applying the ABI clear-sky mask product and correcting biases using a lookup table. In GREMLIN, strong echoes are closely related to the existence of lightning and corresponding low brightness temperatures, which result in different error distributions over different regions of CONUS. This leads to negative biases in cold seasons over Washington State, lower 30-dBZ critical success index caused by high misses over the Northeast, and higher false alarms over Florida that are due to higher frequency of lightning.

Open access
Imme Ebert-Uphoff
and
Kyle Hilburn

Abstract

The method of neural networks (aka deep learning) has opened up many new opportunities to utilize remotely sensed images in meteorology. Common applications include image classification, e.g., to determine whether an image contains a tropical cyclone, and image-to-image translation, e.g., to emulate radar imagery for satellites that only have passive channels. However, there are yet many open questions regarding the use of neural networks for working with meteorological images, such as best practices for evaluation, tuning, and interpretation. This article highlights several strategies and practical considerations for neural network development that have not yet received much attention in the meteorological community, such as the concept of receptive fields, underutilized meteorological performance measures, and methods for neural network interpretation, such as synthetic experiments and layer-wise relevance propagation. We also consider the process of neural network interpretation as a whole, recognizing it as an iterative meteorologist-driven discovery process that builds on experimental design and hypothesis generation and testing. Finally, while most work on neural network interpretation in meteorology has so far focused on networks for image classification tasks, we expand the focus to also include networks for image-to-image translation.

Full access
Kyle A. Hilburn
,
Imme Ebert-Uphoff
, and
Steven D. Miller

Abstract

The objective of this research is to develop techniques for assimilating GOES-R series observations in precipitating scenes for the purpose of improving short-term convective-scale forecasts of high-impact weather hazards. Whereas one approach is radiance assimilation, the information content of GOES-R radiances from its Advanced Baseline Imager saturates in precipitating scenes, and radiance assimilation does not make use of lightning observations from the GOES Lightning Mapper. Here, a convolutional neural network (CNN) is developed to transform GOES-R radiances and lightning into synthetic radar reflectivity fields to make use of existing radar assimilation techniques. We find that the ability of CNNs to utilize spatial context is essential for this application and offers breakthrough improvement in skill compared to traditional pixel-by-pixel based approaches. To understand the improved performance, we use a novel analysis method that combines several techniques, each providing different insights into the network’s reasoning. Channel-withholding experiments and spatial information–withholding experiments are used to show that the CNN achieves skill at high reflectivity values from the information content in radiance gradients and the presence of lightning. The attribution method, layerwise relevance propagation, demonstrates that the CNN uses radiance and lightning information synergistically, where lightning helps the CNN focus on which neighboring locations are most important. Synthetic inputs are used to quantify the sensitivity to radiance gradients, showing that sharper gradients produce a stronger response in predicted reflectivity. Lightning observations are found to be uniquely valuable for their ability to pinpoint locations of strong radar echoes.

Open access
Michael C. Kruk
,
Kyle Hilburn
, and
John J. Marra

Abstract

This study analyzes 25 years of Special Sensor Microwave Imager (SSM/I) retrievals of rain rate and wind speed to assess changes in storminess over the open water of the Pacific Ocean. Changes in storminess are characterized by combining trends in both the statistically derived 95th percentile exceedance frequencies of rain rate and wind speed (i.e., extremes). Storminess is computed annually and seasonally, with further partitioning done by phase of the El Niño–Southern Oscillation (ENSO) index and the Pacific decadal oscillation (PDO) index. Overall, rain-rate exceedance frequencies of 6–8 mm h−1 cover most of the western and central tropical Pacific, with higher values present around the Philippines, Japan, Mexico, and the northwest coast of Australia. Wind speed exceedance frequencies are a strong function of latitude, with values less (greater) than 12 m s−1 equatorward (poleward) of 30°N/S. Statistically significant increasing trends in rain rate were found in the western tropical Pacific near the Caroline Islands and the Solomon Islands, and in the extratropics from the Aleutian Islands down the coast along British Columbia and Washington State. Statistically significant increasing trends in wind speed are present in the equatorial central Pacific near Kiribati and the Republic of the Marshall Islands (RMI), and in the extratropics along the west coast of the United States and Canada. Thus, while extreme rain and winds are both increasing across large areas of the Pacific, these areas are modulated according to the phase of ENSO and the PDO, and their intersection takes aim at specific locations.

Full access
Peter J. Marinescu
,
Daniel Abdi
,
Kyle Hilburn
,
Isidora Jankov
, and
Liao-Fan Lin

Abstract

Estimates of soil moisture from two National Oceanic and Atmospheric Administration (NOAA) models are compared to in situ observations. The estimates are from a high-resolution atmospheric model with a land surface model [High-Resolution Rapid Refresh (HRRR) model] and a hydrologic model from the NOAA Climate Prediction Center (CPC). Both models produce wetter soils in dry regions and drier soils in wet regions, as compared to the in situ observations. These soil moisture differences occur at most soil depths but are larger at the deeper depths below the surface (100 cm). Comparisons of soil moisture variability are also assessed as a function of soil moisture regime. Both models have lower standard deviations as compared to the in situ observations for all soil moisture regimes. The HRRR model’s soil moisture is better correlated with in situ observations for drier soils as compared to wetter soils—a trend that was not present in the CPC model comparisons. In terms of seasonality, soil moisture comparisons vary depending on the metric, time of year, and soil moisture regime. Therefore, consideration of both the seasonality and soil moisture regime is needed to accurately determine model biases. These NOAA soil moisture estimates are used for a variety of forecasting and societal applications, and understanding their differences provides important context for their applications and can lead to model improvements.

Significance Statement

Soil moisture is an essential variable coupling the land surface to the atmosphere. Accurate estimates of soil moisture are important for forecasting near-surface temperature and moisture, predicting where clouds will form, and assessing drought and fire risks. There are multiple estimates of soil moisture available, and in this study, we compare soil moisture estimates from two different National Oceanic and Atmospheric Administration (NOAA) models to in situ observations. These comparisons include both soil moisture amount and variability and are conducted at several soil depths, in different soil moisture regimes, and for different seasons and years. This comprehensive assessment allows for an accurate assessment of biases within these models that would be missed when conducting analyses more broadly.

Open access
Bryan Shaddy
,
Deep Ray
,
Angel Farguell
,
Valentina Calaza
,
Jan Mandel
,
James Haley
,
Kyle Hilburn
,
Derek V. Mallia
,
Adam Kochanski
, and
Assad Oberai

Abstract

Increases in wildfire activity and the resulting impacts have prompted the development of high-resolution wildfire behavior models for forecasting fire spread. Recent progress in using satellites to detect fire locations further provides the opportunity to use measurements towards improving fire spread forecasts from numerical models through data assimilation. This work develops a physics-informed approach for inferring the history of a wildfire from satellite measurements, providing the necessary information to initialize coupled atmosphere-wildfire models from a measured wildfire state. The fire arrival time, which is the time the fire reaches a given spatial location, acts as a succinct representation of the history of a wildfire. In this work, a conditional Wasserstein Generative Adversarial Network (cWGAN), trained with WRF-SFIRE simulations, is used to infer the fire arrival time from satellite active fire data. The cWGAN is used to produce samples of likely fire arrival times from the conditional distribution of arrival times given satellite active fire detections. Samples produced by the cWGAN are further used to assess the uncertainty of predictions. The cWGAN is tested on four California wildfires occurring between 2020 and 2022, and predictions for fire extent are compared against high resolution airborne infrared measurements. Further, the predicted ignition times are compared with reported ignition times. An average Sorensen’s coefficient of 0.81 for the fire perimeters and an average ignition time difference of 32 minutes suggest that the method is highly accurate.

Open access