a. Statistical downscaling and prediction of precipitation

Statistical prediction and downscaling methods for precipitation are based on the extensively studied association between extreme precipitation and recurrent large-scale meteorological patterns (LSMP), which provide favorable environment for smaller-scale processes often underlying the extreme precipitation events (although not all such events are tied to LSMP). Barlow et al. (2019) reviewed, among other things, the types of meteorological synoptic systems and mechanisms for extreme precipitation LSMPs for the North America region and found a great diversity of LSMPs depending on the geographical location and season. LSMPs are distinct from teleconnection patterns in that the LSMPs are conditioned on the occurrence of a specific event—for example, extreme precipitation—whereas classical teleconnections are not. The most intuitive way of defining the LSMP is through compositing, although a variety of other methods are available, including regression-based and cluster-analysis methods (Grotjahn et al. 2016). For example, Robertson et al. (2016) used K-means cluster analysis (Robertson and Ghil 1999) of the reanalysis wind data over North America to identify seven distinct large-scale circulation types and tie some of them to enhanced probability of springtime flooding events in the Midwest of the United States. We note here that while identifying a small subset of large-scale recurrent patterns to classify precipitation-producing weather states is an attractive methodology, it is at odds with the extreme-precipitation LSMPs’ great diversity mentioned above. Hence, the practical utility of such methodologies to downscale precipitation over a large domain such as CONUS is likely to be limited.

Classical regression approaches such as canonical correlation analysis (CCA: Wilks 2011) also have a limited applicability to short-term precipitation modeling due to non-Gaussian and intermittent nature of precipitation. However, they may be suitable and have been utilized for the prediction of seasonal rainfall both directly (Sinha et al. 2013) and as an auxiliary tool for selecting external predictors in conjunction with alternative methodologies (Holsclaw et al. 2016). The most widely used class of the latter alternative methods for statistical modeling, downscaling and prediction of precipitation involves, in one way or another, generalized linear models (GLM; McCullagh and Nelder 1989). GLMs are an extension of classical linear regression models to simulate the conditional expectation of a non-Gaussian distributed variable—such as precipitation—as a function of external predictors referred to as exogenous variables. The latter variables are associated with nonstationary forcing related to the climate variability external to the climate subsystem of interest or, of most relevance to the present discussion, with the occurrence of LSMPs. The GLM models are typically constructed to estimate probability of daily precipitation at a grid point or weather station level (e.g., Furrer and Katz 2007), although some generalizations to multiple stations accounting for spatial correlations between them are also available (Kenabatho et al. 2012). Manzanas et al. (2018) fitted separate GLM models to downscale daily precipitation occurrence and, separately, daily precipitation amount at each grid cell using upper-air predictors simulated by multimodel seasonal climate hindcasts over the Philippines. They showed that this methodology can yield a significant forecast skill improvement for seasonal precipitation prediction over that of raw forecasts in cases where the dynamical model predicts large-scale exogenous variables better than it predicts the precipitation itself.

An alternative approach to precipitation modeling over a spatially extended array of grid points or stations—a hidden Markov model (HMM) approach—assumes the existence of a few discrete “hidden” weather states that capture spatial dependencies of rainfall probabilities within the region considered. The Markovian daily transitions between these states are tied to exogenous predictors via GLM regression; in the latter case these models are referred to as nonhomogeneous hidden Markov models (NHMM; Robertson et al. 2004). Holsclaw et al. (2016) developed a combined HMM-GLM approach, in which a weather state HMM model is complemented by a GLM model that can modify individual hidden states at a station level in response to external predictors, rather than the probabilities of transitions between fixed states, as in the traditional NHMM. We speculate that this approach would also be challenging to adapt for faithful modeling of extreme precipitation over the entire CONUS, where, once again, the heaviest tails of local precipitation distributions are associated with a multitude of precipitation producing systems (Barlow et al. 2019), rather than with a small number of weather states and/or exogenous predictors.

b. Present approaches

To summarize the above discussion, neither classical linear regression-based methods nor clustering or HMM methods are directly suitable for statistical modeling and prediction of precipitation over the entirety of CONUS due to non-Gaussian and intermittent nature of precipitation and a great diversity of precipitation-producing systems/mechanisms in this region, respectively. GLM regression methods may work at a gridpoint level but will still require the choice of exogenous dynamical variables based on a subjective zoning of the area; these methods are also incompatible with automated linear regularization and predictor-selection techniques such as CCA or closely related partial least squares methods (PLS; Wold et al. 1984).

Here we address these difficulties via a new methodology based on statistical modeling of the so-called pseudo-precipitation (PP) field, which uses column integrated water vapor saturation deficit as a negative complement to precipitation (Yuan et al. 2019). Pseudo-precipitation is thus characterized by a more symmetric distribution than the actual precipitation, opening up a possibility of utilizing standard linear regression methods for its modeling. Furthermore, in contrast to classical precipitation field, pseudo-precipitation patterns provide, additionally, information on both the synoptic-scale and anisotropic mesoscale environment, including LSMPs, in which local precipitation occurs, making it ideally suited for linear inverse modeling (LIM; Penland 1996; Penland and Sardeshmukh 1995) and related data-driven modeling methodologies (Kravtsov et al. 2005, 2010, 2016, 2017). The LIMs exhibit subseasonal forecasts skill comparable to that of state-of-the-art numerical weather prediction (NWP) models (see e.g., Winkler et al. 2001) and, most importantly, are able to isolate initial states associated with useful predictability of its own, as well as of NWP-model based forecasts (Newman et al. 2003; Albers and Newman 2019). This property can be helpful for identifying potentially predictable high-impact precipitation events—the main focus of the present study. The proof-of-concept mesoscale-resolving regional inverse models of surface temperature over CONUS have been developed and tested before (Kravtsov et al. 2017); these models are complex enough (yet numerically efficient) to provide an overarching description and forecast utilization of LSMPs associated with local weather extremes. We expect the same statement to be true for the combined surface temperature/pseudo-precipitation modeling we propose here.

In addition to the above purely statistical and numerically efficient methodology, we will also develop and test a procedure for selecting an optimal thinned subsample of representative dates by utilizing the GEFSv12 reforecasts of precipitation for the 2000–19 period. This procedure would allow one to conduct a greatly reduced number of hydrologic hindcasts to estimate the adequacy of the reduced sample for the postprocessing, validation and end-user needs. However, it is much more computationally demanding than the proposed purely data-driven methodology insofar as it still requires, in the first place, the full-blown meteorological reforecasts of the entire climate state to determine the thinned subsample, which somewhat defies the purpose of data thinning. Full, everyday reforecasts were available for the GEFS versions 10 (Hamill et al. 2013) and 12 (Guan et al. 2022), but such full records may not be available in the future to be subsampled. Yet, the present dynamical/statistical ad hoc algorithm based on the Global Ensemble Forecast System version 12 (GEFSv12; Hamill et al. 2022; Zhou et al. 2022) reforecasts can be considered a control against which to evaluate our main statistical modeling methodology, and, in what follows, we describe this algorithm first.

a. Selecting reforecast case dates based on heavy precipitation in GEFSv12 reforecasts

The GEFSv12 reforecasts are retrospective forecasts on a global 0.25° grid, spanning the period 2000–19; they are 5-member ensemble reforecasts generated once per day, from 0000 UTC initial conditions. We argue here that a metric of an event’s extremeness should be based on precipitation magnitude as opposed to, say, the quantile of today’s forecast relative to its climatological distribution (e.g., a 0.1-in. forecast in the desert may be an extreme event relative to the local climatology but still of marginal significance to hydrologic applications). In the present methodology, the importance of a case for potential selection was judged based on the 0–10-day total GEFSv12 ensemble-mean reforecast precipitation P₁₀, sampled daily over the 2000–19 period for each of the 18 enclosed CONUS regions associated with distinct 2-digit Hydrologic Unit Codes (HUC-2 units: https://nas.er.usgs.gov/hucs.aspx). Some case choices were based on large ensemble-mean precipitation averaged over the entire HUC-2 unit, while others were optimized on the top 20% of grid points inside that HUC-2 unit (at the 0.25° resolution) to emphasize smaller-scale impactful events. A small number of cases were also based on large CONUS-wide ensemble-mean precipitation. More specifically, the subjectively chosen breakdown of cases was as follows:

1. 30% of the total cases were optimized based on the maximum 10-day ensemble mean precipitation in that HUC-2 unit. After choosing a case day on this criterion, an ad hoc de-weighting of the day before and the day after was applied so they are less likely to be chosen. However, we find that the algorithm often chooses case days separated by at least 2 days (which can be easily adjusted if desired).

2. 60% of the total cases are optimized based on the maximum 10-day ensemble-mean precipitation at a smaller number of grid points within the HUC. Specifically, for each HUC, precipitation means are sorted at grid points within the HUC, lowest to highest. The mean of the highest 20 sorted points is calculated, and the case days are selected based on the dates with the highest mean values for those 20 points. As with 1 above, an ad hoc de-weighting factor is applied after the selection of a particular case day to surrounding days to make that date less likely to be chosen.

3. The remaining 10% are chosen based on maximal CONUS-averaged ensemble-mean precipitation.

These choices were admittedly arbitrary, but were intended to balance cases selected for impact across large river basins, for flooding in smaller sub-basins, and for overall impact that may cross basins. De-weighting of adjacent days was applied to guard against a tendency for case days to gather around a few significant events, thereby providing less independent samples.

In developing the above merged set of dates from across the subdomains, we chose the first case date from each subdomain unless it was a repeat. Then we proceeded to the second ordered case date in each subdomain, the third, and so forth, until we have reached n total cases, where n is an adjustable predetermined size of the thinned sample. The lists of presumed important cases were developed separately for the warm (April–September) and cool season (October–March), with n = 520, or 1 sample per week.

The resulting procedure produces a list of dates with an irregular sampling in time, which is to be expected if there exist long periods with no hydrologically significant activity, which the algorithm aims to skip to provide more samples when there is strong forcing. The clustering around the largest storms from multiple initial conditions/issued date–times is controlled, to an extent, by our de-weighting procedure. This procedure involves a trade-off: on the one hand, we do not want a lot of shared information between samples; on the other hand, we do want to sample the largest events from several issued date–times and, hence, lead durations. Other adjustable parameters include the total number of cases n and the proportions of cases associated with each of the case categories (1, 2, and 3) above.

We will hereafter refer to the thinned sample produced by the above procedure as sample_A; illustrative examples from this sample will be presented alongside with the results from our alternative, purely data-driven methodology presented below.

b. Selecting reforecast cases using an EMR statistical model

1) Datasets and variables: Introducing pseudo-precipitation

We analyzed data from the National Centers for Environmental Prediction North American Regional Reanalysis (NARR) (http://www.esrl.noaa.gov/psd/data/gridded/data.narr.html); Mesinger et al. (2006), using daily “observations” on a 349 × 277 grid with nominal horizontal resolution of 32 km and 29 pressure levels, over the 1979–2020 period; about a third of these data are from locations over land, leading to ∼30000 data points for each day throughout the 42-yr period and for a single-level field. The NARR dataset has been widely used in the climate downscaling community (see Zobel et al. 2018 and references therein). Bukovsky and Karoly (2007) found that NARR provides faithful estimates of the observed precipitation over CONUS, although some biases exist over Canada due to a relatively poor quality of the assimilated data there.

We utilized NARR datasets for the (daily) accumulated precipitation Pr and 2-m air temperature T_a. We also used the air temperature T and specific humidity Q data at all available pressure levels to compute the air dryness D related to the column-integrated water vapor saturation deficit (Yuan et al. 2019). In an air column of area δA, the mass of water vapor δm to be added to achieve saturation throughout the column is

$\delta m=-\delta A{{\displaystyle \int }}^{\text{}}\left({\rho }_{\upsilon }-{\rho }_{\upsilon ,s}\right)dz=\delta A{{\displaystyle \int }}^{\text{}}\left({\rho }_{\upsilon }-{\rho }_{\upsilon ,s}\right)\hspace{0.17em}\frac{dp}{\rho g}=\frac{\delta A}{g}{{\displaystyle \int }}^{\text{}}\left(Q-Q_s\right)dp.$(1)

Here z is the geometric height, p is the pressure, ρ and ρ_υ are the dry air and water vapor densities, respectively, the subscript s denotes the quantities for saturated air and g = 9.82 m s⁻² is the gravity acceleration. The specific humidity of saturated air Q_s can be computed as (Bolton 1980) follows:

$Q_s=\epsilon \hspace{0.17em}\frac{e_s}{p};\hspace{0.17em}{\mathit{e}}_{\mathit{s}}=6.112\exp \left(\frac{17.67\mathit{T}}{\mathit{T}+243.5}\right),\hspace{0.17em}$(2)

where ε = 0.62198 is the ratio of the molecular weights of water and dry air, e_s is the saturation water vapor pressure, and air temperature T is expressed in degrees Celsius (°C). Air dryness D is defined as the equivalent water depth associated with the quantity δm in (1):

$D=-\frac{\delta m}{{\rho }_w\delta A}=-\frac{1}{{\rho }_{w}g}{{\displaystyle \int }}^{\text{}}\left(Q-Q_s\right)dp,$(3)

where ρ_w = 1000 kg m⁻³ is water density. The air dryness in (3) can be thought of as a negative complement to precipitation and used to construct the so-called pseudo-precipitation field PP, which is, here, equal to the actual precipitation Pr if Pr > 0.001 m day⁻¹ or to Pr + D (essentially, the air dryness D) otherwise.

The PP field incorporates the information about both precipitation, which can exhibit small-scale intermittent structures, and multiscale synoptic environment (see Fig. 1); it thus provides a promising, yet unexplored way to characterize and predict, statistically, wet and dry weather conditions. One of its attractive features is that the distribution of PP, unlike that of Pr, is a single-mode, two-tailed distribution, which makes PP more similar to other dynamical and thermodynamic variables describing atmospheric state. This opens up a possibility for using standard methodologies developed previously for temperature and flow-field analysis and modeling (CCA, LIMs) to analyze and model pseudo-precipitation PP and, hence, the actual precipitation Pr, which equals pseudo-precipitation when PP > 0 and zero otherwise.

(top) Pseudo-precipitation (PP), as well as (bottom) precipitation Pr on 1 Jun 1979, derived from the NARR reanalysis (m). White areas in the bottom plot are either outside of the NARR domain or, otherwise, have zero Pr.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

(top) Pseudo-precipitation (PP), as well as (bottom) precipitation Pr on 1 Jun 1979, derived from the NARR reanalysis (m). White areas in the bottom plot are either outside of the NARR domain or, otherwise, have zero Pr.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

(top) Pseudo-precipitation (PP), as well as (bottom) precipitation Pr on 1 Jun 1979, derived from the NARR reanalysis (m). White areas in the bottom plot are either outside of the NARR domain or, otherwise, have zero Pr.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

a. Using the EMR model as an emulator of daily precipitation evolution

Preliminary inspection of the EMR-model simulations of daily precipitation Pr—which equals PP when PP > 0 and is zero otherwise—reveals model biases in the distribution of precipitation events (not shown). To eliminate these biases, we apply quantile mapping (for each of the DJF, MAM, JJA, and SON seasonal subsets) to each 1979–99 model simulation of pseudo-precipitation to make the simulated local distributions of this quantity identical to those based on the original 1979–99 NARR data. Specifically, the observed 1979–99 and simulated 2000–20 PP time series at a given grid point and for a given season (DJF, MAM, JJA, and SON) were sorted in the ascending order, upon which the sorted 2000–20 simulated values were replaced by the sorted 1979–99 observed values, then put back in the original order (cf. Hamill 2018). This procedure automatically ensures the identical local (i.e., a given grid point’s) precipitation distributions between the model and NARR reanalysis as well. However, the spatiotemporal characteristics of sequences of daily precipitation maps are entirely due to dynamics embedded in the EMR model’s propagator. Examples of such sequences for the warm and cold season are shown in Fig. 2 and supplemental Fig. S4, respectively and give one a visual impression of how well the model matches the space–time structure of the observed stationary and propagating precipitation patterns; the external link to longer sequences is also available in the supplemental information.

A JJA-season sequence of daily surface precipitation maps (m) from (left) arbitrary (random) realization of EMR model and (right) NARR reanalysis. Day “1” in a panel caption would correspond to 1 Jun 1979. White areas in the bottom plot are either outside of the NARR domain or, otherwise, have zero Pr.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

A JJA-season sequence of daily surface precipitation maps (m) from (left) arbitrary (random) realization of EMR model and (right) NARR reanalysis. Day “1” in a panel caption would correspond to 1 Jun 1979. White areas in the bottom plot are either outside of the NARR domain or, otherwise, have zero Pr.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

A JJA-season sequence of daily surface precipitation maps (m) from (left) arbitrary (random) realization of EMR model and (right) NARR reanalysis. Day “1” in a panel caption would correspond to 1 Jun 1979. White areas in the bottom plot are either outside of the NARR domain or, otherwise, have zero Pr.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

We also compute, for future use, daily time series of day 0–10 cumulative precipitation (P₁₀) and display its seasonal means and 99th percentiles in Figs. 3 and 4, respectively. Note that while the simulated local daily precipitation distributions are fixed due to quantile mapping, the simulated and observed distributions of P₁₀ can be different if the spatial scales or persistence/intermittency of the simulated precipitation differ from the observed characteristics. However, this does not seem to be the case here: the simulated P₁₀ seasonal cycle is entirely consistent with observations (Fig. 3); for example, the southeastern United States experiences highest rainfall totals in JJA, while the northwestern United States—in DJF. The EMR simulated P₁₀ biases are small, under 2 mm over the vast majority of grid points, and, on average, are at 2%–3% of climatology (depending on the season) over land (and rarely exceeding 10% of climatology at any given location). We note here again that the EMR model is only trained to accurately model precipitation over land, where the T_a/PP EOFs were computed and is thus not expected to perform equally well over ocean. Statistically significant P₁₀ climatological differences indeed only occur at a small number of grid points either over the ocean or, otherwise, in locations with small precipitation totals.

The 1979–99 seasonal climatology of the 0–10-day total precipitation at the surface—P₁₀ (mm). (left) Climatology based on NARR reanalysis; (center) climatology based on an ensemble of 100 EMR model simulations; and (right) the difference between the simulated and NARR based P₁₀ climatology, with stippling indicating the regions over which this difference is of the same sign for more than 97 realizations (so, effectively, it is statistically significant at the 5% level).

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

The 1979–99 seasonal climatology of the 0–10-day total precipitation at the surface—P₁₀ (mm). (left) Climatology based on NARR reanalysis; (center) climatology based on an ensemble of 100 EMR model simulations; and (right) the difference between the simulated and NARR based P₁₀ climatology, with stippling indicating the regions over which this difference is of the same sign for more than 97 realizations (so, effectively, it is statistically significant at the 5% level).

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

The 1979–99 seasonal climatology of the 0–10-day total precipitation at the surface—P₁₀ (mm). (left) Climatology based on NARR reanalysis; (center) climatology based on an ensemble of 100 EMR model simulations; and (right) the difference between the simulated and NARR based P₁₀ climatology, with stippling indicating the regions over which this difference is of the same sign for more than 97 realizations (so, effectively, it is statistically significant at the 5% level).

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

As in Fig. 3, but for the 99th percentile of P₁₀.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

As in Fig. 3, but for the 99th percentile of P₁₀.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

As in Fig. 3, but for the 99th percentile of P₁₀.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

The simulated P₁₀’s 99th percentile (Fig. 4) is an overestimate compared to observations, including large areas over land, so the EMR model does tend to overpredict the magnitude of extreme events. This reflects, perhaps, overly persistent local precipitation anomalies, but the overall match between the simulated and observed P₁₀ distributions is still fairly reasonable: the P₁₀ 99th percentile patterns are captured extremely well, with spatial correlation coefficient between the EM based and reanalysis patterns exceeding 0.95 for each season, while the seasonal biases in the P₁₀ 99th percentile magnitude over land remain under 25% of its climatology on average (and rarely exceed 50% of the climatology at any given grid point).

b. EMR model predictive skill

To initialize the EMR model forecasts starting from a given day n within the 2000–20 validation interval, we assume that the observable state vectors x at days n, n − 1, and n − 2 are all known. This, however, still requires us to solve for the values of the hidden-level variables r⁽¹⁾ and r⁽²⁾ at the initial day n, which involves two pre-steps of the model (4) driven by a random r⁽³⁾ forcing that ensure dynamical consistency [within the model (4)] of the hidden-state variables with the observables x_n, x_n−1, x_n−2. After these pre-steps, the model is integrated forward in a normal way until the time n + 10. This procedure is repeated for all of the available initial conditions. Upon transformation back to physical space, the collection of PP forecasts for a given lead time is, again, quantile mapped to the 1979–99 local daily PP distributions; finally, zeroing out the negative values of this quantile mapped PP forecast gives the final forecast of the daily precipitation at this lead time, for each initial condition. Summing up the precipitation forecasts for the days n to n + 10 makes up the final P₁₀ forecast for each initial condition; we produced an ensemble of 100 such forecasts under different realizations of the random forcing. Below we will focus on these P₁₀ forecasts when estimating the EMR model’s forecast skill.

1) Deterministic skill

We first discuss some traditional deterministic measures of skill by comparing the observed P₁₀ values with their ensemble-mean EMR based prediction. Figure 5 provides cool-season examples of such a comparison for select cases of substantial observed P₁₀ episodes over CONUS (see Fig. S5 for analogous warm-season comparisons). These cases were arbitrarily selected from the subset of “cases-of-interest” (sample_A) identified by the GEFSv12-based methodology described in section 3a. Visual inspection confirms reasonable EMR forecasts (left column) of the spatial scale, shape, location, and magnitude of the observed large P₁₀ events (center column), qualitatively and quantitatively similar to analogous GEFSv12 forecasts (right column). For example, the EMR model outperforms the GEFSv12 forecast of P₁₀ along the Pacific coast on 11 January 2000 (second row, black box) in better capturing both the pattern and magnitude of precipitation there, whereas GEFSv12 forecast overpredicts/underpredicts precipitation in the southern/northern part of this region, respectively. Both prediction systems over the same region on 26 November 2001 (third row, black box) show a mixed performance: the EMR model captures the pattern, but underpredicts the magnitude of precipitation, while GEFSv12 forecast is much better in terms of the magnitude in the southern part of the region but is essentially the same with the EMR forecast in the northern part of the region, also under-predicting the magnitude of the observed P₁₀. At the same time, the GEFSv12 forecast over the southeastern United States (third row, red box) successfully reproduces a band of heavy precipitation there, which the EMR model completely misses. A few days later though (fourth row, red box), the EMR model matches the magnitude of precipitation better than the GEfSv12 forecast, which overpredicts the maximum precipitation by as much as 50 mm.

Examples of (cool season) P₁₀ forecasts (mm) using (left) EMR model and (right) GEFSv12 system, along with the (center) actual P₁₀ maps based on NARR reanalysis. The forecast initialization time (the same across each row) is shown in panel titles in the YYYYMMDDHH format. The black and red rectangles indicate regions further discussed in text.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Examples of (cool season) P₁₀ forecasts (mm) using (left) EMR model and (right) GEFSv12 system, along with the (center) actual P₁₀ maps based on NARR reanalysis. The forecast initialization time (the same across each row) is shown in panel titles in the YYYYMMDDHH format. The black and red rectangles indicate regions further discussed in text.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

Examples of (cool season) P₁₀ forecasts (mm) using (left) EMR model and (right) GEFSv12 system, along with the (center) actual P₁₀ maps based on NARR reanalysis. The forecast initialization time (the same across each row) is shown in panel titles in the YYYYMMDDHH format. The black and red rectangles indicate regions further discussed in text.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

Despite a reasonable case-by-case performance, the overall correlations between the observed and forecasted P₁₀ time series (for each season) (Fig. 6, left), while positive, are fairly low, at the 0.2–0.3 level in most areas, with the exception of a few season-dependent regions reaching potentially useful levels of 0.5–0.6, including the northwestern United States in MAM/SON, and the monsoon region covering Arizona–New Mexico in JJA. However, these correlations, even when small, are consistently higher than those for the damped-persistence forecasts (Fig. 6, right).

The EMR model precipitation forecast skill. (left) Correlation between (1-day lead time) EMR forecast (ensemble mean of 100 members) and daily P₁₀ time series from NARR reanalysis, for each season. (right) The difference between forecast skill of the EMR model and (daily) damped persistence forecast of P₁₀ (see text for details). Stippling indicates the areas of positive differences, where the EMR forecast outperforms the damped persistence forecast.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

The EMR model precipitation forecast skill. (left) Correlation between (1-day lead time) EMR forecast (ensemble mean of 100 members) and daily P₁₀ time series from NARR reanalysis, for each season. (right) The difference between forecast skill of the EMR model and (daily) damped persistence forecast of P₁₀ (see text for details). Stippling indicates the areas of positive differences, where the EMR forecast outperforms the damped persistence forecast.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

The EMR model precipitation forecast skill. (left) Correlation between (1-day lead time) EMR forecast (ensemble mean of 100 members) and daily P₁₀ time series from NARR reanalysis, for each season. (right) The difference between forecast skill of the EMR model and (daily) damped persistence forecast of P₁₀ (see text for details). Stippling indicates the areas of positive differences, where the EMR forecast outperforms the damped persistence forecast.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

The root-mean-square (rms) distance between the observed and forecasted P₁₀ time series (Fig. 7) is generally close to the P₁₀’s climatological standard deviation (so that the ratio of the two shown in Fig. 7 is close to 1), with EMR model forecasts outperforming damped persistence forecasts in some of the southern areas—in particular, over regions across Mexico except for the DJF season—as well as over the northwestern United States in MAM and SON, consistent with Fig. 6, but performing similar to damped persistence forecasts elsewhere.

EMR model’s P₁₀ forecast (2000–20) root-mean-square (rms) error as a fraction of (1979–99) climatological standard deviations, for each season. Note the inverted color scale; otherwise, the same layout and conventions as in Fig. 6.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

EMR model’s P₁₀ forecast (2000–20) root-mean-square (rms) error as a fraction of (1979–99) climatological standard deviations, for each season. Note the inverted color scale; otherwise, the same layout and conventions as in Fig. 6.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

EMR model’s P₁₀ forecast (2000–20) root-mean-square (rms) error as a fraction of (1979–99) climatological standard deviations, for each season. Note the inverted color scale; otherwise, the same layout and conventions as in Fig. 6.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

Overall, the deterministic measures of skill suggest, at best, a modest performance of the EMR model in forecasting P₁₀. This, however, may be in part due to unsuitability of these measures to describe the forecast quality of a discontinuous and highly intermittent—in space and time—state variable such as precipitation. In particular, considering the ensemble-mean forecast only completely disregards much of the useful information associated with the entire ensemble of forecasts.

2) Probabilistic characteristics of skill

A perhaps more suitable measure of skill for precipitation should involve probabilistic characteristics associated with ensemble forecasts of this quantity. An example of such a measure is shown in Fig. 8, which plots the climatological ratio of the interquartile range (IQR) of the EMR model forecasts to the climatological IQR of P₁₀. This quantity is related to the so-called potential predictability (see Kleeman 2002 and references therein), with the values less than 1 (this value corresponds to climatology forecast) and increasingly closer to zero indicating a progressively more reliable forecast. Based on this measure, the EMR model provides potentially useful forecasts throughout the region of interest, including CONUS, except for the western (warm season) or southwestern (cool season) United States and Mexico, where the EMR predictions are over-dispersed when forecasting the behavior of the North Pacific/Hawaiian high (also see below).

The (average 2000–20) EMR model’s P₁₀ forecast interquartile range (IQR)—based on an ensemble of 100 forecasts—relative to the (1979–99) climatological IQR of P₁₀, for each season. The ratios below unity (stippling) indicate an enhanced forecast utility relative to that of climatology forecast. Note the inverted color scale.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

The (average 2000–20) EMR model’s P₁₀ forecast interquartile range (IQR)—based on an ensemble of 100 forecasts—relative to the (1979–99) climatological IQR of P₁₀, for each season. The ratios below unity (stippling) indicate an enhanced forecast utility relative to that of climatology forecast. Note the inverted color scale.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

The (average 2000–20) EMR model’s P₁₀ forecast interquartile range (IQR)—based on an ensemble of 100 forecasts—relative to the (1979–99) climatological IQR of P₁₀, for each season. The ratios below unity (stippling) indicate an enhanced forecast utility relative to that of climatology forecast. Note the inverted color scale.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

While providing a measure of forecast utility, the potential predictability does not directly compare the forecast with the actual observed precipitation value for the time of forecast. To do so, we here introduce an additional forecast skill measure—the forecast success rate—by counting the frequency of forecasts for which the observed P₁₀ value is within the IQR range of the EMR forecast ensemble. The EMR model forecast success rate has large areas with values exceeding 0.5 (the observed value of P₁₀ is within the IQR of EMR forecasts 50% of the time or more) and sometimes nearing the value of 1 (Fig. 9, left). By this metric, the EMR forecasts seem most reliable off of the Pacific coast, but this is simply an artifact of an over-dispersion documented over the same region in Fig. 8. Combining the results of Figs. 8 and 9, one can argue that the EMR model forecasts have low dispersion and >50% success rates over much of the United States from the Great Plains to California consistently throughout the seasons (or for about 50% of the grid points over the United States). We also combined the damped persistence forecasts of P₁₀ with the mean and IQR range of the corresponding EMR forecast to compute the success rate associated with the damped persistence forecast: in particular, the “range” associated with a damped persistence forecast f_p was set to be f_p − Δ_m, f_p + Δ_p, where Δ_m and Δ_p are the offsets between the EMR model’s ensemble mean and its 25th and 75th percentiles, respectively. We verified that the damped persistence forecast success rate defined in this way is substantially lower than the EMR model’s success rate (Fig. 9, right).

(left) The EMR forecast success rate defined as the fraction of P₁₀ forecasts (over all initial conditions, in each season separately) for which the actual P₁₀ value from the NARR reanalysis is within the IQR of (100-member) ensemble forecasts; stippling shows the areas with success rate exceeding the value of 0.5 (associated with the climatology forecast). (right) The difference between the EMR success rate and the success rate associated with the damped persistence forecast combined with the IQR of the EMR model (see text for details); stippling denotes the areas of positive differences, where the EMR model outperforms damped persistence forecast.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

(left) The EMR forecast success rate defined as the fraction of P₁₀ forecasts (over all initial conditions, in each season separately) for which the actual P₁₀ value from the NARR reanalysis is within the IQR of (100-member) ensemble forecasts; stippling shows the areas with success rate exceeding the value of 0.5 (associated with the climatology forecast). (right) The difference between the EMR success rate and the success rate associated with the damped persistence forecast combined with the IQR of the EMR model (see text for details); stippling denotes the areas of positive differences, where the EMR model outperforms damped persistence forecast.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

(left) The EMR forecast success rate defined as the fraction of P₁₀ forecasts (over all initial conditions, in each season separately) for which the actual P₁₀ value from the NARR reanalysis is within the IQR of (100-member) ensemble forecasts; stippling shows the areas with success rate exceeding the value of 0.5 (associated with the climatology forecast). (right) The difference between the EMR success rate and the success rate associated with the damped persistence forecast combined with the IQR of the EMR model (see text for details); stippling denotes the areas of positive differences, where the EMR model outperforms damped persistence forecast.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

Hence, the EMR model produces reliable, low-dispersion forecasts that tend to track the observed precipitation (signal), much more so than the damped persistence forecasts. Kleeman (2002) argued that a forecast’s relative entropy

$R={\displaystyle \sum }_i{p}_i\ln \frac{p_i}{q_i},$(6)

where p_i is climatological distribution and q_i is that for the prediction, can be very useful in characterizing prediction utility as it naturally captures both the signal and dispersion components of skill. We computed R by estimating the seasonal climatological distributions p_i over the 1979–99 training period for each grid point in terms of the normalized histograms using 20 equally spaced bins between the minimum and maximum P₁₀ value at this grid point, and then computing the distribution q_i of the 100 EMR forecasts using the same bins, for each initial condition in the 2000–20 period. With the natural logarithms in (6), the units of information in which the relative entropy is measured are referred to as nats (as opposed to bits for base-2 logarithms). The relative entropy measures how different the forecast distribution is from a climatological distribution. However, the expectation of R (characterizing climatological difference between forecasts and observations) would tend to be lower for the forecast schemes that are more skillful than others. For example, the climatological relative entropy associated with the damped persistence forecasts is expected to be higher than that for the EMR forecasts. This is indeed the case in Fig. 10 (left), where the relative entropy here was only computed and shown over the grid points at which the 99th percentile of P₁₀ exceeded 50 mm (cf. Fig. 7, left). The basin-mean of the ratios (at each grid point) of the relative entropy associated with the damped persistence forecasts to the climatological R of the EMR forecasts over land exceeds 2.6 for all seasons. Yet, over time, the relative entropy associated with individual P₁₀ forecasts can greatly exceed its climatological value (Fig. 10, right): the analogous basin mean of ratios of the 99th percentile of the EMR forecasts’ relative entropy to the climatological value of R is 5.1, 4.0, 2.9, and 3.4 for the DJF, MAM, JJA, and SON seasons, respectively. In section 4c below, we will develop a subsampling strategy in which the forecasts with large values of the quantity R are tagged to define and sample potential large and extreme precipitation events.

Relative entropy of EMR forecasts (nats). (left) The expectation (climatology), with stippling showing the areas where this expectation exceeds that associated with the damped persistence forecast (see text for details); (right) the 99th percentile. Note that the relative entropy here was only computed and shown over the grid points at which the 99th percentile of P₁₀ exceeded 50 mm (cf. Fig. 7, left); the areas in which this is not the case are colored white.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Relative entropy of EMR forecasts (nats). (left) The expectation (climatology), with stippling showing the areas where this expectation exceeds that associated with the damped persistence forecast (see text for details); (right) the 99th percentile. Note that the relative entropy here was only computed and shown over the grid points at which the 99th percentile of P₁₀ exceeded 50 mm (cf. Fig. 7, left); the areas in which this is not the case are colored white.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

Relative entropy of EMR forecasts (nats). (left) The expectation (climatology), with stippling showing the areas where this expectation exceeds that associated with the damped persistence forecast (see text for details); (right) the 99th percentile. Note that the relative entropy here was only computed and shown over the grid points at which the 99th percentile of P₁₀ exceeded 50 mm (cf. Fig. 7, left); the areas in which this is not the case are colored white.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

c. EMR-based probabilistic algorithm for thinning reforecast sample size

Note that the cases displayed in Figs. 5 and S5 were selected using the ad hoc algorithm based on heavy precipitation in GEFSv12 reforecasts (section 3a; sample_A). The multipage image files with analogous maps for other selected cases are available through a web page referenced in the supplemental material. As mentioned before, the same algorithm was applied to the EMR model’s ensemble-mean P₁₀ forecasts, which are also available through the supplementary website; see a brief discussion at the end of this section.

We here also developed and applied an alternative strategy, which selects the dates based on the large value of the EMR forecasts’ relative entropy. In particular, we computed, for each day, the average among the top 10% relative-entropy gridpoint values over CONUS. The grid points over which this was done were also preselected to have the seasonal 1979–99 P₁₀ 99th percentile exceeding 5 cm, thus excluding the white areas in Fig. 10. Each day in the record was then ranked based on its relative entropy score. To eliminate possible effects of any long-term relative entropy trends, we selected 40% of the highest-score dates from the first and 40% of the highest-score dates from the second half of the original 2000–20 sample. We next edited out the member with a higher R from all the pairs of consecutive high-relative-entropy days identified above, and then from the pairs separated by two days. This procedure results in the identification of 1095 cases separated by at least two days out of the total 7671 days comprising the 2000–20 period. We argue that this subset is an optimal subset, which includes the majority of the high-impact events, while being also representative of the climatological P₁₀ distribution. If more frequent sampling is required, the additional dates for reforecasts can be added at random from the remainder of the record.

The size of the latter sample is also consistent with that of the sample_A, which has 520 cases per each of the semiannual cool and warm seasons (over 2000–19 period, with the following breakdowns: DJF—188, MAM—276, JJA—247, and SON—329 cases). The corresponding breakdowns for the present sample are DJF—282, MAM—290, JJA—240, and SON—283 cases, featuring a more uniform seasonal distribution of cases, with more DJF cases and fewer SON cases compared to the GEFSv12 based subsample. The two samples turn out to be largely independent, with only 198 (∼20%) matching dates over the 2000–19 period. A few examples of the P₁₀ observed and predicted maps based on the present sample are shown in Figs. 11 and S6, and others are available through the supplementary material. The third column of these figures shows the distribution of the EMR forecasts’ relative entropy on a given day, which tends to track the areas of large and extreme precipitation (recall that the relative-entropy-based selection criterion was only applied over CONUS, rather than over a larger region of the NARR reanalysis).

Examples of (left) (cool season) P₁₀ forecasts (mm) using EMR model and (center) the actual P₁₀ maps based on NARR reanalysis. (right) The corresponding map of the relative entropy. The forecast initialization time (the same across each row) is shown in panel titles in the YYYYMMDDHH format. Note that, similar to Fig. 10, the relative entropy in the right column plots was only computed and shown over the grid points at which the 99th percentile of P₁₀ exceeded 50 mm (cf. Fig. 7, left); the areas in which this is not the case are colored white.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Examples of (left) (cool season) P₁₀ forecasts (mm) using EMR model and (center) the actual P₁₀ maps based on NARR reanalysis. (right) The corresponding map of the relative entropy. The forecast initialization time (the same across each row) is shown in panel titles in the YYYYMMDDHH format. Note that, similar to Fig. 10, the relative entropy in the right column plots was only computed and shown over the grid points at which the 99th percentile of P₁₀ exceeded 50 mm (cf. Fig. 7, left); the areas in which this is not the case are colored white.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

Examples of (left) (cool season) P₁₀ forecasts (mm) using EMR model and (center) the actual P₁₀ maps based on NARR reanalysis. (right) The corresponding map of the relative entropy. The forecast initialization time (the same across each row) is shown in panel titles in the YYYYMMDDHH format. Note that, similar to Fig. 10, the relative entropy in the right column plots was only computed and shown over the grid points at which the 99th percentile of P₁₀ exceeded 50 mm (cf. Fig. 7, left); the areas in which this is not the case are colored white.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

To assess relative performance of the two methods, we computed distributions of P₁₀ associated with each sample and compared them with the climatological distribution of P₁₀. An example of these distributions in Fig. 12 demonstrates that the EMR based sample provides a better match to the NARR based P₁₀ climatological distribution than the GEFSv12 based subsample, which tends to be excessively heavy tailed: DJF panel gives a particularly clear example of this for the location chosen. The positive bias of the GEFSv12 based sample—perhaps natural, given the selection criterion built on the direct occurrence of the large or extreme precipitation—is also evident in the maps of the climatological mean (Fig. 13) and, to a somewhat lesser extent, in the maps of the 99th percentile (Fig. S7) of the distributions based on the full and subsampled data. Overall, the present sample has a distribution of 0–10-day total precipitation that is closer to the distribution based on the full data compared to that of GEFSv12 based sample_A, while capturing the majority of high-impact precipitation events. It should be noted, however, that the ultimate test of the success of the subsampling will be the accuracy of postprocessed precipitation guidance based on the sample at hand, and not the fidelity against the NARR data. For example, heavier precipitation periods preferentially sampled by the GEFSv12 algorithm by design may be particularly important for establishing the statistical relationships in situations with heavy precipitation that are of greatest interest.

The P₁₀’s probability density function (PDF) estimates at 47.4°N, 122.4°W (Seattle, WA) based, for each season, on the entire daily P₁₀ data (blue), and two thinned subsamples ∼1/7 the size of the whole available data: a subsample based on relative entropy of EMR forecasts (EMR-RE) (sample, red) and the one (sample_A, yellow) based on ensemble-mean GEFSv12 P₁₀ forecasts associated with significant precipitation events over CONUS (see text for details).

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

The P₁₀’s probability density function (PDF) estimates at 47.4°N, 122.4°W (Seattle, WA) based, for each season, on the entire daily P₁₀ data (blue), and two thinned subsamples ∼1/7 the size of the whole available data: a subsample based on relative entropy of EMR forecasts (EMR-RE) (sample, red) and the one (sample_A, yellow) based on ensemble-mean GEFSv12 P₁₀ forecasts associated with significant precipitation events over CONUS (see text for details).

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

The P₁₀’s probability density function (PDF) estimates at 47.4°N, 122.4°W (Seattle, WA) based, for each season, on the entire daily P₁₀ data (blue), and two thinned subsamples ∼1/7 the size of the whole available data: a subsample based on relative entropy of EMR forecasts (EMR-RE) (sample, red) and the one (sample_A, yellow) based on ensemble-mean GEFSv12 P₁₀ forecasts associated with significant precipitation events over CONUS (see text for details).

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

The differences between the estimates of P₁₀’s climatological mean based on (left) EMR-based thinned sample and the entire seasonal P₁₀ data; (center) GEFSv12-based thinned sample_A and the entire seasonal P₁₀ data; and (right) EMR-based thinned sample and GEFSv12-based thinned sample_A. Stippling shows areas where the differences are statistically significant at the 5% level according to the two-sided bootstrap test involving surrogate random subsamples of the same size as either the EMR-based thinned sample or GEFSv12-based thinned sample_A.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

The differences between the estimates of P₁₀’s climatological mean based on (left) EMR-based thinned sample and the entire seasonal P₁₀ data; (center) GEFSv12-based thinned sample_A and the entire seasonal P₁₀ data; and (right) EMR-based thinned sample and GEFSv12-based thinned sample_A. Stippling shows areas where the differences are statistically significant at the 5% level according to the two-sided bootstrap test involving surrogate random subsamples of the same size as either the EMR-based thinned sample or GEFSv12-based thinned sample_A.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

The differences between the estimates of P₁₀’s climatological mean based on (left) EMR-based thinned sample and the entire seasonal P₁₀ data; (center) GEFSv12-based thinned sample_A and the entire seasonal P₁₀ data; and (right) EMR-based thinned sample and GEFSv12-based thinned sample_A. Stippling shows areas where the differences are statistically significant at the 5% level according to the two-sided bootstrap test involving surrogate random subsamples of the same size as either the EMR-based thinned sample or GEFSv12-based thinned sample_A.

Citation: Weather and Forecasting 37, 5; 10.1175/WAF-D-21-0162.1

• Download Figure
• Download figure as PowerPoint slide

Finally, we note here that the thinned sample_B obtained using the same algorithm as for the GEFSv12 data, but applied to the EMR precipitation forecasts, produced results inferior of those associated with either the EMR-RE sample or the GEFSv12-based sample_A in terms of the similarity of climatological precipitation distributions based on the thinned and full available data samples (Figs. S8 and S9). This may be due to the fact that the EMR forecasts of P₁₀ have a smaller deterministic skill than analogous high-end GEFSv12 reforecasts.