1. Introduction
Surface meteorological analyses provide an essential foundation for research, applications, and resource management across various disciplines, including atmospheric and climate science, hydrology and water resources, and ecology. These analyses are used for various purposes, including evaluating atmospheric and climate model performance, as forcing data for land surface or hydrologic models, and as a basis for climate variability and trend analysis. Observed surface meteorological field products for precipitation and temperature over large domains, such as the contiguous United States, have usually been developed by research centers and public agencies and, until recently, have been deterministic. Over the last several decades, there has been incremental improvement in these products, in the form of increases in spatiotemporal resolution and extent, multisensor data fusion, and accessibility (Kitzmiller et al. 2013).
Large-domain hydrometeorological datasets are inherently uncertain, however, due to several well-known factors: 1) measurement errors in both in situ and remotely sensed inputs and errors in model-based analyses (such as reanalyses); 2) errors in remote sensing retrieval algorithms; and 3) inadequate sampling in space and time, which necessitates various forms of interpolation to the target space–time resolution and undermines the accuracy of fusion methods. Errors in meteorological analysis products can propagate into any application that may utilize these data. As such, there is a need to quantify the errors in surface meteorological fields and create probabilistic or ensemble meteorological analyses that provide estimates of dataset uncertainty. Such uncertainty estimates are invaluable as a basis for assigning confidence to analyses, such as for storm event rainfall or an extreme temperature estimate, and as a driver for uncertainty estimates in related geophysical variables. Land model data assimilation, for example, depends on the ability to quantify land surface state uncertainty, a portion of which arises from uncertainty in meteorological inputs (Wagener and Gupta 2005).
Several types of deterministic large-domain (i.e., regional to global) surface analyses exist that serve different communities, each with various strengths and weaknesses. For example, many widely used national-scale datasets within the United States rely primarily on in situ station observations and achieve a relatively high grid resolution through climatologically aided interpolation methods (Maurer et al. 2002; Livneh et al. 2015; Xia et al. 2012). To avoid large and uneven spatial gaps between stations and account for terrain influences, the interpolation approaches impose known surface climate tendencies such as orographic influences on precipitation and lapsing of temperature with elevation. To achieve this, several of the datasets rely on the terrain–climate relationship represented in the climatology of the Precipitation-Elevation Regressions on Independent Slopes Model (PRISM) dataset (Daly et al. 1994, 2008). Other national scale datasets rely on station–radar fusions ranging from relatively simple radar bias correction techniques in the NOAA Multisensor Precipitation Estimation (MPE) algorithm (Fulton et al. 1998; Lin and Mitchell 2005) to the complex fusion algorithms within the Multi-Radar Multi-Sensor (MRMS) product (Zhang et al. 2016, 2014). Several quasi-global satellite-based datasets incorporate station observations and are at a high enough resolution to be useful for applications, including PERSIANN-CCS, which is optimized for extreme events (Hong et al. 2004; Hsu et al. 2013); IMERG, which incorporates Global Precipitation Measurement mission active radar information (Huffman et al. 2020); and CMORPH (Joyce et al. 2004), which is a blend of passive microwave and infrared satellite information.
Recent research has advanced new methods for uncertainty estimation and the generation of ensemble surface meteorological fields (Clark and Slater 2006; Kirstetter et al. 2015; Newman et al. 2015; Cornes et al. 2018; Newman et al. 2019, Newman et al. 2020; Tang et al. 2021) that have direct useability in land data assimilation and ensemble forecasting applications (Huang et al. 2017; Ren et al. 2019; Slater and Clark 2006). Ensemble forcing products are now available for various domains, including the entire contiguous United States (CONUS) and across Europe. The Gridded Meteorological Ensemble Tool (GMET) methodology, applied over the CONUS and the basis of the present work, is based on multiple logistic and linear regression using static geophysical attributes to predict precipitation and temperature across a grid (Clark and Slater 2006; Newman et al. 2015). Regression errors are used to condition spatially correlated Gaussian random fields for ensemble generation. GMET has recently also been extended to include ensemble climatologically aided interpolation (eCAI) (Newman et al. 2019) as well as ensemble wetting loss and wind undercatch corrections (Newman et al. 2020). The E-OBS ensemble (Cornes et al. 2018), available across Europe, uses methods similar to GMET and also leverages CAI by using monthly values to inform a deterministic daily generalized additive model. The EMDNA (Ensemble Meteorological Dataset for North America; Tang et al. 2021) wraps the static-predictor version of GMET into a multidataset optimal interpolation (fusion) framework to help quantify uncertainty and generate forcing ensembles.
The GMET spatial regression approach represents well-known terrain influences on surface climate. Even so, limitations arise when the sample of observations to define the regression relationship is inadequate or when the terrain influence is not a dominant driver of spatial variability in precipitation or temperature. The former limitation can occur when the station density is sparse; in this case, assembling a robust sample of observations to estimate regression parameters requires enlarging the domain from which they are taken beyond a credible representation of terrain influencing a single coherent weather event. The latter limitation can occur when advection, synoptic flow, or convection drives the spatial pattern of observations in a small region. In both instances, useful information may exist in ancillary datasets, including concurrent simulations from an atmospheric model with sufficient spatial resolution to resolve mesoscale surface circulation features, remotely sensed data, radar observations, or gridded reanalyses. For example, Beck et al. (2019) and Tang et al. (2021), as noted above, extract information from multiple reanalysis datasets to improve their final surface meteorological datasets. A range of considerations, such as the available datasets and user appetite for complexity, may govern the adoption of one method over another. In summary, however, the opportunity to enhance the GMET regression directly to harness such additional information for spatial meteorological estimation motivates the present study. An advantage of such an approach is that it will inherently adjust for systematic magnitude biases in the ancillary gridded observations because their utilization will depend on pattern correlation within a given time step. Our specific hypothesis is that augmenting the static terrain predictors currently used in GMET with time-varying predictors derived from the operational NOAA High-Resolution Rapid Refresh (HRRR) model (Benjamin et al. 2016) will improve the ability of the GMET spatial regression to estimate the spatial distribution and amount of precipitation.
The following sections describe the case study used in the experiment, the observational and model datasets involved, the GMET approach, and the experiment results. Finally, we conclude with a discussion of the findings and potential further work.
2. Ensemble forcing generation methodology
The GMET spatial regression algorithm has been applied to a diverse range of climate conditions across the Colorado Rockies (Clark and Slater 2006), CONUS (Newman et al. 2015), and outside CONUS (OCONUS) across Hawaii and Alaska (Newman et al. 2019, 2020). As noted in section 1, the goal of the current work is to extend the static terrain predictor fields by including temporally varying predictors in the regression framework. This effort necessitates adding an internal cross-validation capability, and modifications to GMET were undertaken to both understand and improve its selection of predictors and local weighting strategy. Details of the GMET development are given in appendix B.
a. GMET spatial regression with static predictors
The spatial regression approach for interpolating in situ meteorological observations uses spatially distributed information as predictor fields in an ordinary least squares (OLS) linear regression to explain the spatial distribution of point in situ observations. In prior applications of GMET, the spatial predictors have been static geophysical attributes (directional slopes, elevation, latitude, and longitude). The regression was applied to predict daily precipitation, mean temperature, and diurnal temperature range (DTR) for each target grid cell, on each day, based on the current observed values of those variables within a sample from the 30 nearest meteorological stations and given their relationship to the local terrain features at the station locations. This strategy generated dynamic (time-varying) uncertainty estimates that are driven by daily observed meteorological conditions.
Daily temperature and DTR are often modeled as variables with normal or near-normal distributions, whereas precipitation is intermittent at short time scales, leading to a probability distribution function that may contain many zero values and if so, must be handled differently. This is commonly achieved by defining one distribution for the probability of occurrence of zero precipitation, po, and a second distribution for the amounts (values) of all nonzero precipitation. Prior to use in the spatial regression, precipitation and temperature data are normalized. We adopt Gaussian distributions for daily mean temperature and the DTR and do not transform the data before use in the spatial regression. In contrast, daily precipitation amounts are subjected to a parametric Box–Cox (Box and Cox 1964) transformation before normalization (details are given in appendix A).
In the prior applications of GMET, the regression has been localized by incorporating distance-dependent weights for each station in the samples pooled for each grid cell, set to 30 stations as in Clark and Slater (2006). For the current application, an option is introduced to allow for equal station weighting. This change was motivated by recognizing that such additional localization undermines the ability to estimate regression coefficients in small sample situations, and further introduces noise in the regression relationship because the influence of terrain on meteorological conditions is not an isotropic function of distance, but rather derives both from nonisotropic terrain features and weather event features.
Ensemble realizations of precipitation and temperature are generated in GMET by sampling from the estimated distributions of YP, XT, and XD at all valid grid points and time steps using spatiotemporally correlated random fields for which the spatial, temporal, and cross-variable correlation structure has been determined from station observations, as was done in Clark and Slater (2006) and Newman et al. (2015).
b. Extension to include time-varying predictors
As noted above, GMET is extensible to include other predictors. Currently, GMET uses static topographical predictors as inputs to the multiple linear regression equations [Eqs. (1) and (2)] to develop precipitation and temperature distributions. These include latitude, longitude, elevation, west–east slope, and north–south slope, which are derived from the digital elevation model (DEM) for the domain of interest. These attributes are also calculated and supplied for the station locations, with slope terms assigned from the terrain grid values from the grid cell in which the station is located. Here we extend GMET to include both temporally varying (dynamic) predictors and the ability to drop static predictors by modifying Ag and As [Eq. (3)], where the spatial predictors a1 through
Schematic showing the GMET version 2.0 procedure. Superscript numerals indicate the following: 1) The dynamic predictors need to be regridded to the GMET input terrain features grid as a preprocessing step. 2) The station data must be serially complete. Station data preprocessing methods are explicitly documented in the appendix of Newman et al. (2015). Square boxes contain method elements, and slanted boxes contain data elements.
Citation: Journal of Hydrometeorology 23, 7; 10.1175/JHM-D-21-0193.1
3. Experimental setting and data
The study methods were developed and evaluated for a case study domain spanning California, focusing on a flooding event in February 2017 that impacted Oroville Reservoir, which controls the outflow of the Feather River basin (FRB) in California. The experimental setting and datasets are described below.
a. The precipitation event in the context of Oroville climatology
California’s water year 2017 was exceptional (White et al. 2019), with two atmospheric river (AR) events in early February causing record-setting precipitation. From 2 to 11 February 2017, over 600 mm fell in the FRB upstream of Oroville Dam. Early season (October–December) precipitation events meant that the main spillway of Oroville Dam was already in operation to control the high lake level. During the second AR event from 6 to 11 February, the FRB received the larger portion of the event precipitation, with the greatest daily totals on 7 February. The increased inflow to Lake Oroville damaged the main spillway on 7 February, and on 9 February the emergency spillway was used for the first time in the dam’s history. The extreme nature of this precipitation event posed a forecasting challenge for dam operators, and the predicted inflow to Lake Oroville was underestimated (Henn et al. 2020).
In the online supplemental material, we show the synoptic conditions for 1200 UTC 7 February 2017, using maps of NESDIS blended total precipitable water (TPW), TPW anomaly from a 1988–99 climatology, and a 700-mb (1 mb = 1 hPa) pressure level map. The AR event brings an atypical amount of moisture from the tropics to the western United States. At 1200 UTC 7 February, northeastern California had over 3 times the climatological amount of total precipitable water. Southwesterly flow at 25 m s−1 transports moisture over the Sierra Nevada, raising the 700-mb level dewpoint temperatures over 2.2°C. The orographic enhancement of convection over the Sierra Nevada led to extreme precipitation falling on near-saturated soils. The event’s high freezing levels further raised the runoff efficiency of the storm; both snowmelt and excessive runoff contributed to the historic inflow to Lake Oroville from 7 to 10 February.
b. Meteorological station data
Station data ingested into GMET are extracted from the Global Historical Climatology Network–Daily dataset (GHCN-Daily; Menne et al. 2012). The GHCN-Daily data originate from a variety of networks, including the Cooperative Observer (COOP) network; the U.S. Natural Resources Conservation Service (NRCS) Snowpack Telemetry (SNOTEL) network; the Community Collaborative Rain, Hail, and Snow (CoCoRaHS) network; the Automated Weather/Surface Observing Systems (AWOS/ASOS); and the Remote Automated Weather Stations (RAWS) network. Station precipitation data used to evaluate GMET are taken from the NOAA Meteorological Assimilation Data Ingest System (MADIS; https://madis.ncep.noaa.gov; Miller et al. 2005). The MADIS data contain observations from NOAA (METAR, maritime, NEPP, CRN) and non-NOAA (mesonet) networks. The MADIS dataset is exclusively made from METAR (20) and mesonet (236) sites in our study domain, and data from sites also found in the GHCN-Daily dataset were excluded. While temperature observations from the METAR network are assimilated into the HRRR model, the focus of this study is on precipitation estimation and METAR precipitation is not assimilated into the HRRR model (Benjamin et al. 2016). Hourly precipitation accumulations are summed to daily totals. Figure 2 shows a map of the GHCN and MADIS sites used and the rectangular domain of stations influencing the FRB.
Map of GHCN stations (172, blue markers) used as input to GMET and MADIS stations (256, orange markers) for evaluation. The location of Oroville Dam is marked with a white star, and the Feather River basin (FRB) watershed is outlined in black. The 19 MADIS stations within the FRB (red markers) show a subset of the MADIS evaluation dataset with a direct hydrological link to Lake Oroville.
Citation: Journal of Hydrometeorology 23, 7; 10.1175/JHM-D-21-0193.1
c. Numerical weather prediction data
This paper introduces a general ability in GMET to use time-varying gridded predictors. Information from any dynamic ancillary source can now be used, including satellite-based weather datasets, reanalyses that fuse observations and model data, or analyses or predictions from NWP systems. We test this new capability by using the HRRR, a cloud-resolving, convection-permitting atmospheric model run operationally in real time by NOAA at a 3-km horizontal resolution, with hourly updates and an 18-h forecast lead time. The 3-km grids initialize HRRR with 3-km radar assimilation, performed every 15 min over a 1-h period. We concatenate 1-h lead forecast fields (i.e., the first hour of the forecast) into an hourly analysis and aggregate the resulting gridded time series to the space–time resolution of the GMET application (1/16°, daily time step) for several candidate predictor variables.
As an example of this data, Fig. 3 shows the 1200 UTC hourly analysis from the HRRR for the AR event on 7 February 2017. The HRRR model reproduces the AR event well with precipitable water values between 30 and 40 mm, matching the value reported in the NESDIS product (see Figs. S1 and S2 in the online supplemental material). The southwesterly flow transports moisture between a low pressure disturbance to the northwest and an anticyclone to the southwest of California. Because these meteorological factors influenced the severity of the Oroville case more than terrain, the good representation of synoptic-scale moisture advection achieved with the HRRR suggests it will contain useful dynamic predictor fields.
HRRR analyses data show the synoptic-scale conditions at 1200 UTC 7 Feb 2017. Precipitable water (mm; colored shading), mean sea level pressure (hPa; black numbered contours), and storm motion wind (m s−1; barbs).
Citation: Journal of Hydrometeorology 23, 7; 10.1175/JHM-D-21-0193.1
To investigate promising predictor variables for precipitation, we calculate Spearman rank-order correlation coefficients for a set of candidate HRRR variables versus the station precipitation observations input to GMET. Figure 4 shows a time series of these correlation coefficients for each day of the precipitation event. Three candidate predictor variables—total precipitation (prcp), vertically integrated liquid water (vilw), and relative humidity with respect to precipitable water (rhpw)—have positive correlations above 0.5, suggesting potential usefulness. Other variables assessed included u-/υ-component storm motion (uvstm) and u-/υ-component wind at 10 m (uv10m). Total precipitation has the highest and most consistent correlation, and the other candidates have collinearity to total precipitation, which motivates the choice to use the HRRR total precipitation as a dynamic predictor. Using precipitation as a predictor is a challenge due to its intermittent nature and highly skewed nonzero distribution. Therefore, we first cube root transform precipitation to reduce the skew of the precipitation distribution. Intermittency can lead to a singular matrix (without a unique solution) during regression computations; thus, GMET automatically removes dynamic precipitation as a predictor (for a specific grid cell and day) if this condition is encountered.
Time series showing daily correlation coefficients of candidate HRRR predictor variables (see legend) to daily station precipitation (stn) from the 172 GHCN stations used to force GMET.
Citation: Journal of Hydrometeorology 23, 7; 10.1175/JHM-D-21-0193.1
Determining the best predictors for temperature followed the same approach as for precipitation without transformation. The correlation coefficient for the HRRR variable 2 m temperature versus station temperature observations was approximately 0.9 each day of the event, and it was found to be the most useful candidate HRRR variable to use as a predictor of temperature.
4. Results of the GMET(stn, HRRR) product
To test the expanded dynamic predictor capability of GMET, we generated and compared two datasets, 1) GMET(stn) and 2) GMET(stn, HRRR). Both GMET configurations estimate precipitation at each grid cell using the nearest 40 GHCN-Daily station precipitation and temperature observations in a maximum search distance of 100 km. Local station weights have been used previously to rank predictor stations based on their distance from the target grid cell; here, however, we used equal weights due to the relatively small storm footprint and the correspondingly small sample size used for each regression. Local weights can lead to effective station sample sizes of fewer than 10 stations for fitting the spatial regression, which can fail to recognize more extensive and potentially skillful patterns in HRRR. Sensitivity tests on nominal station sample size (30, 40, 50), search distance (100, 150, 200 km), weighting versus nonweighting, and different k-fold settings (10%, 15%, 20% of data left out) were also performed though not shown, leading to the configuration that was used here.
GMET(stn) uses terrain predictors in the multiple linear regression to estimate precipitation and temperature. GMET(stn, HRRR) adds to the static predictors by including HRRR precipitation and 2 m temperature as dynamic predictors. For both datasets, k-fold cross validation uses nt = 10 for the number of folds (trials), meaning 36 stations are used to train and 4 stations to test. Fifty ensemble members are generated from the regression output for both datasets.
We also assess the quality of the HRRR 0-h lead forecast precipitation field, HRRR(prcp), so that we can understand the components of the GMET(stn, HRRR) product. We first compare these three gridded datasets, then perform an out-of-sample evaluation against MADIS observations from a deterministic (section 4b) and probabilistic perspective (section 4c).
a. Comparison of GMET(stn, HRRR) to GMET(stn)
A comparison of the precipitation accumulation for the entire precipitation event leading to the Oroville dam crisis, 2–11 February 2017, shows that in general, across the domain, GMET(stn, HRRR) produces more precipitation compared to GMET(stn) (Fig. 5). In the FRB particularly, the largest precipitation totals are found where the terrain is steepest, directly upstream of the Oroville Dam. Figure 5c shows that in this area of steepest terrain GMET(stn, HRRR) accumulates >150 mm more precipitation compared to GMET(stn). Figure 5d shows the ability of HRRR to resolve mesoscale precipitation features since it captures the orographic enhancement of precipitation on the upwind side of the Sierra Nevada.
Spatial plots of the ensemble mean precipitation output from GMET for the whole Oroville Dam crisis precipitation event, 2–11 Feb 2017. (a) GMET(stn) is compared to (b) GMET(stn, HRRR), (c) their differences GMET(stn, HRRR) − GMET(stn), and (d) the HRRR precipitation field. A basin mean difference is shown below the color bar in (c).
Citation: Journal of Hydrometeorology 23, 7; 10.1175/JHM-D-21-0193.1
b. Deterministic evaluation
To determine whether the differences between the two datasets shown in Fig. 5 lead to a reduction in estimation error, we assess the performance of the gridded datasets at the MADIS evaluation sites within the FRB, which were not used in the forcing generation. Figure 6a compares ranked values (e.g., the CDF) for each gridded dataset versus the values of the HRRR dataset and the two GMET datasets. Differences are small at lower precipitation totals (<25 mm day−1), with a slight wet bias versus MADIS observations in both GMET datasets. For much of the observed range of precipitation (25–100 mm day−1), the GMET(stn, HRRR) estimates a value between GMET(stn) and HRRR. At higher totals (>100 mm day−1), GMET(stn, HRRR) often estimates a value closer to HRRR than GMET(stn), showing that GMET assigns a higher weight to the HRRR values compared to the GHCN-Daily station values at these stations. Owing to the good precipitation performance of the HRRR in this event, while slightly drier than the out-of-sample MADIS values, including the HRRR information as a dynamic predictor enables GMET(stn, HRRR) to estimate precipitation values that are closer to the observations than GMET(stn) does.
Figure 6b uses the same values to show a pairwise comparison and supports a similar interpretation. In general, the orange GMET(stn, HRRR) markers fall to the right of the blue GMET(stn), showing greater estimated precipitation totals that better match the MADIS observations. Below 75 mm day−1 differences are difficult to discern, but above this, the GMET(stn, HRRR) verification is consistently better than GMET(stn). Figure 6c shows the mean absolute errors (MAEs) of the gridded products binned by the magnitude of the MADIS observation. Below 75 mm day−1, the differences in MAE for the two GMET datasets are similar. However, greater separation of the MAEs is evident above 75 mm day−1. For the two bins with the greatest rates (100–150, 150–200 mm day−1), GMET(stn, HRRR) yields a >5 mm day−1 reduction in MAE compared to GMET(stn), though, still within the 95% confidence interval.
Evaluation of precipitation CDFs (a) from stations in the FRB using MADIS observations and shows a pairwise comparison (b) for the precipitation event. 19 stations in the FRB at daily resolution gives an N = 171. (c) Mean absolute errors from the same evaluation stations in the FRB binned by the magnitude of the MADIS observation (for each bin N = 84, 35, 24, 13, 12, 2). Error bars show the 95% confidence interval.
Citation: Journal of Hydrometeorology 23, 7; 10.1175/JHM-D-21-0193.1
We also calculate the MAE of gridded versus observed precipitation for the whole month of February (not shown) and found that both GMET datasets had similar relative error characteristics. The added value from using the HRRR precipitation as a dynamic predictor is most evident when the performance of HRRR precipitation is good; for example, in the Oroville event, the HRRR resolves mesoscale features like storm-specific gradients that a sparse network of stations would struggle to capture. However, on longer time scales, the often uneven performance of HRRR precipitation makes improvements from GMET(stn) more difficult to distinguish. Given this, there may be a need to enhance GMET to allow for more flexible and dynamic (time–space varying) predictor selection approaches, leveraging the new cross-validation capability and extending the current prescribed static-predictor configuration.
c. Probabilistic evaluation
The analyses above have focused on the ensemble mean precipitation, yet GMET is distinctive in its ability to generate probabilistic (ensemble) forcings. Here, we generate and evaluate the 50 ensemble members’ probabilistic skill at the MADIS sites using reliability diagrams (Fig. 7). GMET(stn, HRRR) shows consistent reliability compared to GMET(stn) at 12.5- and 25-mm thresholds (Figs. 7a,b). Reliability curves from 500 random samples (faded lines) often encompass the one-to-one line. Therefore, we could expect an even closer agreement between estimated and observed probability with a larger sample. Increasing the precipitation event threshold to 50 mm, as seen in Fig. 7c, the spread of the reliability curves generated from the 500 random samples increases. Because the spread is particularly pronounced at ensemble probabilities > 0.5, undersampling effects may be involved here. Points approaching the skill/no skill shading limit indicate some overconfidence or loss of resolution in the ensemble. At the 50-mm threshold, undersampling features are evident in both GMET(stn) and GMET(stn, HRRR) with considerable divergence of the GMET(stn) curves from a straight line. Both GMET(stn) and GMET(stn, HRRR) have a slight wet bias for the higher probabilities of occurrence at this threshold. With this sample, we cannot create reliability diagrams for higher thresholds due to the small number of observed events at higher thresholds. Even so, comparisons indicate that introducing HRRR precipitation as a dynamic predictor may slightly improve GMET’s ability to estimate probabilities of occurrence at higher precipitation values. Overall, however, the statistics between GMET with and without HRRR are similar, with the Brier skill score loss, for example, being relatively consistent for both datasets.
Reliability diagrams for three precipitation thresholds (a) 12.5, (b) 25, and (c) 50 mm. Ensembles of precipitation are taken from all 256 evaluation sites for the whole precipitation event 2–11 Feb 2017. We take an N = 2000 random sample from each dataset (see legend), construct a reliability curve, and repeat 500 times (light shading). The markers and lines with a black edge color show the reliability curve created using the whole dataset N = 2256. The MADIS observed probability at each threshold is shown as a dashed line on the y axis.
Citation: Journal of Hydrometeorology 23, 7; 10.1175/JHM-D-21-0193.1
This analysis has focused on a particular event and watershed, and while we cannot easily infer how the findings might extend to a much larger and more varied hydroclimate domain, we can use a different method to assess whether the HRRR inclusion leads to a significant difference in GMET performance at higher totals (>50 mm day−1) in this case study. Figure 8a shows CDFs of daily precipitation values at the MADIS evaluation sites (Fig. 2) for the entire precipitation event. The distribution of ensemble mean precipitation from GMET(stn, HRRR) is similar to GMET(stn) at values <25 mm day−1 but at values >50 mm day−1 GMET(stn, HRRR) shifts closer to the HRRR distribution. This shift is because the dynamic predictor (HRRR precipitation) is the dominant attribute [a in As for Eq. (3b)] in the regression calculation at these evaluation sites. In these instances, the HRRR precipitation correlates highly to the input station observations meaning that GMET gives it the highest weighting factor, thus making it the most influential predictor. Crucially, the shift toward the HRRR distribution means the GMET(stn, HRRR) distribution moves closer to the MADIS observations than GMET(stn), particularly at higher precipitation values > 50 mm day−1. We can draw 95% confidence intervals for each GMET CDF of ensemble mean precipitation using the ensemble members (Fig. 8b). Parts of the distribution (50–100 mm day−1) show GMET(stn, HRRR) CDF overlapping the GMET(stn) CDF by less than 5% (Fig. 8c). Additionally, computing the Kolmogorov–Smirnov statistic on GMET(stn) versus GMET(stn, HRRR) yields a statistic of 0.04 and a p value of 0.02. We therefore reject the null hypothesis that GMET(stn) and GMET(stn, HRRR) are drawn from the same population.
(a) CDFs of the ensemble mean precipitation values at the MADIS evaluation sites for the whole precipitation event 2–11 Feb 2017. See legend for dataset labels; N = 2274 for each dataset. (b) The 95% confidence interval for CDFs shown in (a). (c) A zoomed portion of (b).
Citation: Journal of Hydrometeorology 23, 7; 10.1175/JHM-D-21-0193.1
5. Summary and conclusions
In this study, we describe an effort to extend the probabilistic forcing generation method outlined in Newman et al. (2015) and Clark and Slater (2006), based on local spatial regression, by 1) augmenting static terrain predictors with concurrent time-varying predictors from NWP (HRRR) surface analyses, and 2) incorporating cross validation in the uncertainty estimation process. These methodological extensions were associated with and tested through enhancements to the GMET software and analysis of a case study focusing on the Feather River basin during a flood event that caused significant damage to the Oroville Reservoir spillway in California. The enhanced method provides a strategy for fusing meteorological gauge observations to ancillary gridded estimates, whether from NWP models, reanalyses, or satellite-derived observations.
The rain event that caused the Oroville Dam crisis provided a useful case study to assess the benefits of merging dynamic and static predictors to generate an ensemble forcing dataset—GMET(stn, HRRR)—as compared to the original GMET(stn) dataset using static terrain predictors only. GMET’s new cross-validation capability was used in the generation of each dataset, both to ensure that the uncertainty estimates (which are manifested in ensemble spread) represent prediction uncertainty and not merely regression calibration uncertainty, and to inform predictor selection in a way that will minimize the potential for regression overtraining.
The case study results lead to a number of useful findings. First, the addition of dynamic predictors in the GMET framework improves the precipitation estimation distribution closer to out-of-sample MADIS observations for the extreme case study event, confirming our study hypothesis. Second, the upslope terrain of the Sierras generally exerts a strong topographic influence on weather systems, and this is true in the Feather River basin directly upstream of Oroville Dam where we see the greatest differences in precipitation from GMET(stn, HRRR) compared to the control simulation. Third, the largest differences from GMET(stn, HRRR) were found at higher precipitation totals (above 100 mm day−1) where GMET(stn, HRRR) significantly reduced mean absolute errors from out-of-sample MADIS observations compared to GMET(stn). Fourth, incorporating HRRR precipitation in GMET also did not degrade its ability to estimate the probability of occurrence at different precipitation thresholds. Note that this study does not assess the question of whether HRRR-based forcings alone or with some form of postprocessing may outperform GMET analyses in a general sense. In this particular, extreme case study, it was notable that HRRR precipitation verified better than GMET against the MADIS observations, but it is unclear whether this outcome would hold more generally. Future investigation may find that HRRR alone is an optimal selection for certain applications, as well as applications for which the GMET capability to fuse station observations with a longer-term dynamic predictor dataset (such as a reanalysis) and generate an arbitrarily large ensemble is more attractive.
The research and development effort represented by this study involved significant exploration of the methodological choices confronted in applying the extended spatial regression technique. These included the decisions regarding localization, the sample sizes used in the regression and cross validation, and the search radius for station sample selection. Clearly, performance tradeoffs exist across the choices—e.g., between using a leave 10% out strategy for cross validation (10 training iterations are needed) versus using a leave 20% out (5 iterations required)—but the selection of sample size is also dependent on the strength of the relationship between predictors (static and dynamic) and observations. The search radius is also a consequential choice because the static versus dynamic predictors may have a greater or lesser spatial agreement with observations (hence predictive skill) depending on the spatial window over which that relationship is assessed. In addition, the localization decision (in which distant stations are given lower weights in the regression) can substantially reduce the effective sample size for regression training and thereby undermine GMET’s ability to give value to a predictor that may have stronger spatial agreement over a larger sampling domain. Extending the GMET model prompts the recognition of a number of such configuration issues, indicating that a priori expertise or objective guidance may be valuable to apply GMET optimally for a given domain, given the variability in factors such as terrain influences on meteorology, station density, and dynamic predictor skill. Accordingly, options have been added to the GMET configuration file to facilitate experimentation with these choices.
The work presented in this paper represents a notable upgrade to the capabilities previously afforded in GMET. In addition to correcting bugs in the prior GMET software, the work improved the normalization of predictors and now allows GMET to be used with a wide array of potential predictors, and with proper cross validation, which is critical to the accurate estimation of forcing ensemble spread. Importantly, the GMET ensembles now reflect the true predictive uncertainty of the chosen statistical model versus just the model calibration uncertainty. Further development and assessment of this spatial regression approach is nonetheless warranted. Due to the scope of this study, the selection of predictors from HRRR was exploratory versus comprehensively data driven, and improved predictor screening and selection approaches could be developed and incorporated into GMET. The time–space domain of the application could be extended to give a more comprehensive sense of the value of ancillary `dynamic observations in GMET. Both HRRR and prior GMET datasets are CONUS-wide, and operational applications of GMET already exist; hence it would be feasible to investigate a CONUS-wide operational GMET–HRRR implementation. In addition, other types of ancillary observations, including from radar, reanalyses, and satellite retrievals, could be tested in this fusion framework, and other forms of spatial regression, including nonlinear formulations, could be incorporated. The improved performance indicated by the results presented here, together with strong performance in existing uses of GMET in large-domain real-time and retrospective applications, suggest that the extended spatial regression approach offers a promising new strategy for probabilistic surface forcing estimation across a wide range of geospatial and environmental applications.
Acknowledgments.
The U.S. Army Corps of Engineers Responses to Climate Change program provided the primary support for this work, and the U.S. Bureau of Reclamation provided additional funding under Cooperative Agreement R16AC00039 that supported the participation of NCAR coauthors. This material is also based upon work supported by the National Center for Atmospheric Research, which is a major facility sponsored by the National Science Foundation under Cooperative Agreement 1852977. We gratefully acknowledge high‐performance computing support from Cheyenne (http://10.5065/D6RX99HX) provided by NCAR’s Computational and Information Systems Laboratory, sponsored by the National Science Foundation.
Data availability statement.
The GMET software is freely available from the Github repository https://github.com/NCAR/GMET. The repository includes the case study example of this paper for using GMET, supporting with the functionality described herein, with all input data required to reproduce the experiment.
APPENDIX A
Transformation and Normalization of Precipitation and Temperature Data
The input data normalization and regression fitting component of GMET requires seven parameters (the parameters of the normal distribution and λ), which are saved in an output file for use in the second GMET component, which performs the ensemble generation and back-transformation to the variables at full amplitude.
APPENDIX B
GMET Development Summary
GMET has undergone steady development since the latest FORTRAN-based software was initially refactored in 2015 (Fig. B1). The developments described in this paper were implemented as an augmentation to the 2015 version, and also included structural, data input/output and usability upgrades made to support operational use as part of the NCAR System for Hydromet Analysis, Research and Prediction (SHARP), some of which is described in Mendoza et al. (2017). Collectively these now form the master branch of the GMET Github repository (https://github.com/NCAR/GMET), which is tagged as GMET version 2.0, the latest release at time of publication. The 2019 and 2020 GMET extensions (e.g., ECAI) are currently maintained in a separate GMET branch.
Development flowchart for the Gridded Meteorological Ensemble Tool (GMET) from the initial GMET concept, Clark and Slater (2006), through the current development (this paper, GMET v2.0).
Citation: Journal of Hydrometeorology 23, 7; 10.1175/JHM-D-21-0193.1
REFERENCES
Beck, H. E., E. F. Wood, M. Pan, C. K. Fisher, D. G. Miralles, A. I. J. M. Van Dijk, T. R. McVicar, and R. F. Adler, 2019: MSWep v2 Global 3-hourly 0.1° precipitation: Methodology and quantitative assessment. Bull. Amer. Meteor. Soc., 100, 473–500, https://doi.org/10.1175/BAMS-D-17-0138.1.
Benjamin, S. G., and Coauthors, 2016: A North American hourly assimilation and model forecast cycle: The Rapid Refresh. Mon. Wea. Rev., 144, 1669–1694, https://doi.org/10.1175/MWR-D-15-0242.1.
Box, G. E. P., and D. R. Cox, 1964: An analysis of transformations. J. Roy. Stat. Soc., 26, 211–252, https://doi.org/10.1111/J.2517-6161.1964.TB00553.X.
Clark, M. P., and A. G. Slater, 2006: Probabilistic quantitative precipitation estimation in complex terrain. J. Hydrometeor., 7, 3–22, https://doi.org/10.1175/JHM474.1.
Cornes, R. C., G. van der Schrier, E. J. M. van den Besselaar, and P. D. Jones, 2018: An ensemble version of the E-OBS temperature and precipitation data sets. J. Geophys. Res. Atmos., 123, 9391–9409, https://doi.org/10.1029/2017JD028200.
Daly, C., R. P. Neilson, and D. L. Phillips, 1994: A statistical-topographic model for mapping climatological precipitation over mountainous terrain. J. Appl. Meteor., 33, 140–158, https://doi.org/10.1175/1520-0450(1994)033<0140:ASTMFM>2.0.CO;2.
Daly, C., M. Halbleib, J. I. Smith, W. P. Gibson, M. K. Doggett, G. H. Taylor, J. Curtis, and P. P. Pasteris, 2008: Physiographically sensitive mapping of climatological temperature and precipitation across the conterminous United States. Int. J. Climatol., 28, 2031–2064, https://doi.org/10.1002/joc.1688.
Fulton, R. A., J. P. Breidenbach, D. J. Seo, D. A. Miller, and T. O’Bannon, 1998: The WSR-88D rainfall algorithm. Wea. Forecasting, 13, 377–395, https://doi.org/10.1175/1520-0434(1998)013<0377:TWRA>2.0.CO;2.
Henn, B., K. N. Musselman, L. Lestak, F. M. Ralph, and N. P. Molotch, 2020: Extreme runoff generation from atmospheric river driven snowmelt during the 2017 Oroville Dam spillways incident. Geophys. Res. Lett., 47, e2020GL088189, https://doi.org/10.1029/2020GL088189.
Hong, Y., K. L. Hsu, S. Sorooshian, and X. Gao, 2004: Precipitation estimation from remotely sensed imagery using an artificial neural network cloud classification system. J. Appl. Meteor., 43, 1834–1853, https://doi.org/10.1175/JAM2173.1.
Hsu, K., S. Scott, N. Phu, B. Dan, and C. Wei, 2013: G-WADI PERSIANN-CCS GeoServer for extreme precipitation event monitoring. Sci. Cold Arid. Reg., 5, 6–15, https://doi.org/10.3724/SP.J.1226.2013.00006.
Huang, C., A. J. Newman, M. P. Clark, A. W. Wood, and X. Zheng, 2017: Evaluation of snow data assimilation using the ensemble Kalman filter for seasonal streamflow prediction in the western United States. Hydrol. Earth Syst. Sci., 21, 635–650, https://doi.org/10.5194/hess-21-635-2017.
Huffman, G. J., and Coauthors, 2020: Integrated Multi-satellite Retrievals for the Global Precipitation Measurement (GPM) Mission (IMERG). Satellite Precipitation Measurement, V. Levizzani et al., Eds., Springer, 343–354, https://doi.org/10.1007/978-3-030-24568-9_19.
Joyce, R. J., J. E. Janowiak, P. A. Arkin, and P. Xie, 2004: CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. J. Hydrometeor., 5, 487–503, https://doi.org/10.1175/1525-7541(2004)005<0487:CAMTPG>2.0.CO;2.
Kirstetter, P.-E., J. J. Gourley, Y. Hong, J. Zhang, C. L. Saber Moazamigoodarzi, and A. Arthur, 2015: Probabilistic precipitation rate estimates with ground-based radar networks. Water Resour. Res., 51, 1422–1442, https://doi.org/10.1002/2014WR015672.
Kitzmiller, D., D. Miller, R. Fulton, and F. Ding, 2013: Radar and multisensor precipitation estimation techniques in National Weather Service hydrologic operations. J. Hydrol. Eng., 18, 133–142, https://doi.org/10.1061/(ASCE)HE.1943-5584.0000523.
Lin, Y., and K. E. Mitchell, 2005: The NCEP Stage II/IV hourly precipitation analyses: Development and applications. 19th Conf. on Hydrology, San Diego, CA, Amer. Meteor. Soc., 1.2, https://ams.confex.com/ams/Annual2005/techprogram/paper_83847.htm.
Livneh, B., T. J. Bohn, D. W. Pierce, F. Munoz-Arriola, B. Nijssen, R. Vose, D. R. Cayan, and L. Brekke, 2015: A spatially comprehensive, hydrometeorological data set for Mexico, the U.S., and southern Canada 1950–2013. Sci. Data, 2, 150042, https://doi.org/10.1038/sdata.2015.42.
Maurer, E. P., A. W. Wood, J. C. Adam, and D. P. Lettenmaier, 2002: A long-term hydrologically based dataset of land surface fluxes and states for the conterminous United States. J. Climate, 15, 3237–3251, https://doi.org/10.1175/1520-0442(2002)015<3237:ALTHBD>2.0.CO;2.
Mendoza, P. A., A. W. Wood, E. Clark, E. Rothwell, M. P. Clark, B. Nijssen, L. D. Brekke, and J. R. Arnold, 2017: An intercomparison of approaches for improving operational seasonal streamflow forecasts. Hydrol. Earth Syst. Sci., 21, 3915–3935, https://doi.org/10.5194/hess-21-3915-2017.
Menne, M. J., I. Durre, R. S. Vose, B. E. Gleason, and T. G. Houston, 2012: An overview of the global historical climatology network-daily database. J. Atmos. Oceanic Technol., 29, 897–910, https://doi.org/10.1175/JTECH-D-11-00103.1.
Miller, P., M. Barth, L. Benjamin, D. Helms, M. Campbell, J. Facundo, and J. O’Sullivan, 2005: The Meteorological Assimilation and Data Ingest System (MADIS): Providing value-added observations to the meteorological community. 21st Conf. on Weather Analysis and Forecasting/17th Conf. on Numerical Weather Prediction, Washington, D.C., Amer. Meteor. Soc.,P1.95, https://ams.confex.com/ams/WAFNWP34BC/techprogram/paper_98637.htm.
Newman, A. J., and Coauthors, 2015: Gridded ensemble precipitation and temperature estimates for the contiguous United States. J. Hydrometeor., 16, 2481–2500, https://doi.org/10.1175/JHM-D-15-0026.1.
Newman, A. J., M. P. Clark, R. J. Longman, E. Gilleland, T. W., Giambelluca, and J. Arnold, 2019: Use of daily station observations to produce high-resolution gridded probabilistic precipitation and temperature time series for the Hawaiian Islands. J. Hydrometeor., 20, 509–529, https://doi.org/10.1175/JHM-D-18-0113.1.
Newman, A. J., M. P. Clark, A. W. Wood, and J. R. Arnold, 2020: Probabilistic spatial meteorological estimates for Alaska and the Yukon. J. Geophys. Res. Atmos., 125, e2020JD032696, https://doi.org/10.1029/2020JD032696.
Ren, H. L., and Coauthors, 2019: The China multi-model ensemble prediction system and its application to flood-season prediction in 2018. J. Meteor. Res., 33, 540–552, https://doi.org/10.1007/s13351-019-8154-6.
Slater, A. G., and M. P. Clark, 2006: Snow data assimilation via an ensemble Kalman filter. J. Hydrometeor., 7, 478–493, https://doi.org/10.1175/JHM505.1.
Tang, G., M. Clark, S. M. Papalexiou, A. Newman, A. Wood, D. Brunet, and P. Whitfield, 2021: EMDNA: Ensemble meteorological dataset for North America. Earth Syst. Sci. Data, 13, 3337–3362, https://doi.org/10.5194/essd-2020-303.
Wagener, T., and H. V. Gupta, 2005: Model identification for hydrological forecasting under uncertainty. Stoch. Environ. Res. Risk Assess., 19, 378–387, https://doi.org/10.1007/s00477-005-0006-5.
White, A. B., B. J. Moore, D. J. Gottas, and P. J. Neiman, 2019: Winter storm conditions leading to excessive runoff above California’s Oroville Dam during January and February 2017. Bull. Amer. Meteor. Soc., 100, 55–70, https://doi.org/10.1175/BAMS-D-18-0091.1.
Xia, Y., and Coauthors, 2012: Continental-scale water and energy flux analysis and validation for North American Land Data Assimilation System project phase 2 (NLDAS-2): 2. Validation of model-simulated streamflow. J. Geophys. Res., 117, D03109, https://doi.org/10.1029/2011JD016051.
Zhang, J., Y. Qi, C. Langston, B. Kaney, and K. Howard, 2014: A real-time algorithm for merging radar QPEs with rain gauge observations and orographic precipitation climatology. J. Hydrometeor., 15, 1794–1809, https://doi.org/10.1175/JHM-D-13-0163.1.
Zhang, J., and Coauthors, 2016: Multi-Radar Multi-Sensor (MRMS) quantitative precipitation estimation: Initial operating capabilities. Bull. Amer. Meteor. Soc., 97, 621–638, https://doi.org/10.1175/BAMS-D-14-00174.1.
