1. Introduction

The source of most atmospheric rainwater is the sea, with rain forming when large droplets eventually become heavy enough to fall to the ground. Rainfall over land eventually flows back to the sea, completing the cycle (Brutsaert 2005). Water is vital to life, but also immensely destructive: understanding the movement of atmospheric water is critical. Forecasting rainfall is therefore important in many disciplines, for example, economics and finance, hydrology, meteorology, ecology, agriculture, and renewable energy.

The U.K. climate is temperate and strongly influenced by the oceans, with cool summers and mild winters (Barry and Chorley 2003). Rainfall forecasting has taken on new urgency in the United Kingdom due to recent flooding caused by extreme rainfall: evidence exists that such extremes may increase in frequency with global temperature increases (Easterling et al. 2000). The Thames River runs directly through London and several major midland and southern towns. Flooding on this river has significant costs to the U.K. economy, and insurance premiums have increased substantially because of these recent severe events.

Rainfall forecasts can therefore help quantify the risk of floods and droughts with which to price products such as flood and crop insurance, weather derivatives, and other commodities (Cao et al. 2004; Diebold et al. 1998; Taylor and Buizza 2006). All forecasts have errors because of the combined uncertainty in observational data and model structure. Comprehensive quantification of this forecast uncertainty is critical to risk assessment, motivating interest in density forecasting producing a distribution over all possible future rainfall events, rather than a single-point forecast misrepresenting these uncertainties. Density forecasts are particularly flexible, allowing the calculation of the probability of any event of interest, such as the probability of occurrence of rain or of extreme rainfall above any threshold. Complementing density forecast comparisons between models, we can also issue the density median as a point forecast.

Numerical weather predictions (NWPs) are highly complex, nonlinear systems producing a single or a set (ensemble) of point forecasts, allowing the anticipation of distinct meteorological events. Statistical time series models are mathematically simple and produce full-density forecasts capturing statistical properties of the data. NWP now routinely outperforms purely statistical methods for medium-range (1 day to 1 week ahead) operational forecasts, but for very short term (a few hours) and very long term (greater than 10 days ahead), statistical approaches remain competitive (Wilks 2006).

Density prediction for product pricing requires accurate forecasts at specific locations, for a wide range of forecast lead times, for which unified approaches to site-specific density forecasts seamlessly covering short- to long-range time scales are needed. For product pricing applications, medium-range global forecasts are currently more useful than short-range regional forecasts, because short-range forecasts do not have sufficiently long forecast horizons.

Because the primary application is the precision quantification of the probability of rain rather than the anticipation of particular meteorological events, dynamical and statistical forecasting, although fundamentally different in approach, can be combined to produce accurate, site-specific density forecasts. Statistical postprocessing (ensemble calibration) (Applequist et al. 2002; Wilks 2006) is one such combination approach.

Precipitation measurement is mostly by ground-based rain gauge measurement of total rainfall depth (Upton et al. 2005). Strong evidence exists that total rainfall amount distributions are discontinuous at zero depth (no rainfall) motivating separate modeling of occurrence (rain/no rain) and intensity (nonzero amount; Cao et al. 2004; Grunwald and Jones 2000; Wilks 1998). These separate models are combined in a mixture density of daily rainfall totals. Similarly, rainfall is nonnegative and non-Gaussian. A wide range of proposed statistical rainfall probability models could be used and it is instructive to test as many as possible. This includes powerful generalized linear models (GLMs; Grunwald and Jones 2000) that allow flexible, nonlinear, non-Gaussian regression, but any new method must demonstrate performance superiority over existing, simpler approaches before being considered successful.

To build time series density forecast models, we explore spatiotemporal correlation and seasonality properties that could be captured and hence exploited. Here we analyze the Thames Valley time series and construct Markov chain GLMs incorporating information from neighboring gauges and past time steps (Grunwald and Jones 2000). We test these models against simple benchmarks, and ensemble NWP forecasts postprocessed with GLMs.

Our main contribution here is to conduct direct tests of a range of methods proposed in the existing literature for producing site-specific, full-density forecasts against novel GLM methods, and hence to suggest an improved NWP postprocessing method. Our investigations explain the performance of these different methods in terms of the autocorrelation properties of the data and the structure of each model, for full-density and point forecasts.

The paper is organized as follows. Section 2 reviews the current state of rainfall measurement, modeling, and forecasting. Section 3 describes the data used and details the correlation structure analysis. Section 4 describes model construction and forecast performance comparison methods used in this study. Section 5 discusses the results of the forecast comparison, and finally section 6 summarizes the paper and concludes with the relevance of these results for future site-specific rainfall forecasting.

2. Review of rainfall measurement and forecasting methods

Ground-based rain gauges capture precipitation, recording the total amount as rainfall depth, usually in millimeters (Upton et al. 2005). The temporal measurement resolution can be high, but accuracy can be site dependent and unreliable in extreme weather and can include melted snow or hail in addition to rain. Radar measurements, by contrast, detect low-altitude atmospheric water content and have excellent spatiotemporal resolution, but current geographic coverage is restrictive and the historical record is short (Upton et al. 2005).

Rainfall forecasting models depend on the application. Thunderstorms implicated in flash floods typically take place on scales of minutes to hours (Battan 1984), requiring forecasts on the shortest time scales. Localized flooding often occurs when medium to heavy rain falls in the same location over several days, inundating rivers and urban drains, requiring forecasts from hours to days. Predicting droughts requires forecasts on longer time scales of weeks to months.

NWP solves equations of atmospheric dynamics and produces rainfall predictions. Calibrated against atmospheric measurements, they vary in spatial scale from synoptic (on the order of 1000 km) to mesoscale (approximately 50 km); current operational models have minimum resolutions of approximately 1.3 km (limited-area mesoscale), forecasting days to a few weeks ahead (Buizza 2003). Sophisticated NWP systems generate an ensemble of predictions by varying the model initial conditions and/or by varying physical parameterization schemes. The frequency distribution of the ensembles estimates the probability density (Buizza 2003; Buizza et al. 1998; Molteni et al. 1996; Palmer et al. 1993). The high computational complexity of running many different parallel forecasts limits most operational NWP systems to single-point forecasts or low spatial resolution ensembles, and important precipitation sources such as isolated thunderstorms and small-scale details are not well resolved.

Classical statistical forecasting identifies relationships between past observations and their temporal successors, using observations at the current forecast origin as predictors for the future state of the atmosphere based solely on these relationships and not on explicit meteorological information (Wilks 2006). Methods include conditional climatology (issuing successors of past observational data closest to the current state as a forecast for the future state), and applications of more sophisticated multiple nonlinear regression such as neural networks (Moura and Hastenrath 2004). Included in this category are statistical time series models that can forecast at the spatiotemporal resolution of rainfall measurements, and are univariate or multivariate (comprising a vector of rainfall measurements from a number of sites simultaneously), producing density forecasts. They are either temporally unconditional or conditional on past time steps.

Statistical postprocessing methods such as the analog method or model output statistics (Applequist et al. 2002; John 2003; Wilks 2006) determine the statistical relationships between forecast NWP variables and actual observations, acting as postprocessors to improve NWP forecasts.

Unconditional daily rainfall models are commonly split into occurrence and intensity (Cao et al. 2004; Grunwald and Jones 2000; Wilks 1998), with the occurrence often modeled as a Bernoulli random variable (Grunwald and Jones 2000). Intensity models usually use exponential family distributions, for example, gamma densities (Grunwald and Jones 2000; Hyndman and Grunwald 2000), exponential mixtures (Cao et al. 2004; Wilks 1998), or truncated normals (Sanso and Guenni 1999). In nonparametric methods, kernel density techniques can model intensity (Cao et al. 2004).

For conditional occurrence models, first-order Markov chains are common (Cao et al. 2004; Wilks 1998). For conditional intensity, generalized linear and generalized additive Markov chain regression models (GLM/GAM) have been used (Grunwald and Jones 2000; Hyndman and Grunwald 2000).

3. Data

The data comprises daily rainfall depth measurements from all 295 Met Office Integrated Data Archive System (MIDAS) WADRAIN rain gauges in the Thames Valley, United Kingdom, within the square grid 51°–52.5°N, −2°–0.5° E, covering an area approximately 400 km by 400 km. Observations cover the time period from 2004 to late September 2007. Figure 1 shows selected examples of the rainfall time series.

Spatiotemporal correlation structure for all 295 gauges is tested. However, only a few of the sites have sufficiently complete records overlapping the available NWP ensemble forecasts, so that, in forecast comparisons, a much smaller subset (10 sites) of this data was selected (Fig. 2). These sites are chosen as a compromise between minimizing the number of missing observations, economic relevance (Heathrow in London), and hydrological interest (Brize Norton received some of the highest rainfall totals during the recent flooding). Table 1 lists the location and number of rainfall observations available for these selected sites (the inset in Fig. 5 shows their physical layout). All gauges have missing measurements and consistent procedures, described below for each forecast model, ensure fair comparisons.

In this section we explore the spatiotemporal correlations in the data. We denote the total rainfall amount on day n = 1, 2, … , N for each site m = 1, 2, … , M as x_n^m, where N is the maximum length in days of the time series, and M = 10 is the number of selected sites for modeling. We denote occurrence with the indicator variable q_n^m, which is 0 for dry days and 1 for days where rainfall x_n^m is nonzero.

We first calculate the autocorrelation function, and the standard Bartlett 95% confidence intervals. Although the autocorrelation function tests the dependence up to second-order statistical moments, such highly non-Gaussian rainfall data could have nonzero higher-order moments. A more general test of independence at different time lags is the time-delayed mutual information (TDMI; Kantz and Schreiber 2004; Little et al. 2006):

View Expanded

Marginal densities P(x_n^m), and joint densities are estimated using histograms of x_n^m. We assume that x_n^m are weakly stationary stochastic processes; then only the relative time lag τ is important and we can assume that are the same. The joint density is estimated using histograms formed by counting the number of times that x_n^m falls into the same histogram bin as . The integrals are approximated using summations. TDMI significance tests use the null hypothesis of no mutual information using bootstrap independent, identically distributed (i.i.d.) time series, generated by randomly permuting individual days’ measurements and destroying any original temporal ordering. The TDMI for each bootstrap is compared with that of the original series to try to reject the null hypothesis at each time lag. If, for a significance probability of α = 0.05, on generating 2/α − 1 = 39 bootstraps, the TDMI at any time lag of the original data is either the smallest or the largest value among all the bootstraps, then we can reject the null hypothesis at that time lag.

Although using histograms to estimate densities is simplistic, we are only interested in the TDMI relative to the i.i.d. bootstraps, and, as such, these inaccuracies are relatively unimportant to the question of whether significant nonlinear/non-Gaussian temporal dependence exists in the time series.

For the spatial correlation analysis of total rainfall amount, the pairwise correlation coefficients for all 295 locations are plotted against physical distance between sites. The no-correlation null hypothesis uses the standard asymptotic normal distribution of Fisher’s z transformation of the correlation coefficient to estimate the p value, tested at 5% significance. Finally, for spatial correlations in occurrence, we calculate the pairwise conditional probability of nonoccurrence of rain for all 295 locations. We take each pair of locations, calculate the conditional probability of nonoccurrence at one location, given that rainfall did/did not occur at the other, and plot these probabilities against physical distance between locations. We can then see the contribution of occurrence of rainfall to the spatial correlation of the total amount.

Turning to the results of this correlation analysis, Table 1 shows that, any one day taken at random, is as likely to be dry as wet. This is to be expected for this generally temperate climate, and the average rainfall intensity is small. Regarding spatial correlation, from Fig. 3, all locations show similar rainfall patterns, and the maximum correlation falls slowly with increasing distance. At any given distance, there is an apparent maximum and minimum correlation between gauges. Figure 4 shows that this effect is even stronger for the nonoccurrence. This is confirmation that most rainfall events are on the meso-alpha scale: resulting from widespread cloud cover due to warm fronts or convective complexes spread over hundreds of square kilometers. Similarly, if dry at one location, then it is highly likely to be dry at all the other locations in the catchment. This physical effect provides some justification for modeling techniques that try to capture catchment-wide spatial cross correlations (Fig. 5).

Regarding temporal correlation, Fig. 6 shows autocorrelation decaying extremely rapidly, and although significant up to 4 days ahead, the TDMI in Fig. 7 shows a slightly different story with significant mutual information to only 3 days. Similar results were obtained for the other selected gauges. The results show that the rapid decay of autocorrelation or TDMI is not just due to linearity limitations of the autocorrelation function; weather systems move rapidly across the United Kingdom and normally dissipate within a few hours. However, the lack of obvious annual periodicity is in need of explanation. Because of lowering temperatures, all other things being equal, lower saturation vapor causes higher average and maximum U.K. rainfall totals during winter (Brutsaert 2005) and increasing frequency of extremes in early autumn. Figure 2 shows some seasonal variation in the rainfall intensity for each month, but this variation is very slight. This slight seasonality is not the strict repetitiveness detectable by autocorrelation and TDMI—exact annual pattern start/end days are ill-defined, varying from year to year. For statistical modeling, rapid exponential decay of mutual information justifies models with very short memory, and regressing on past rainfall at annual time lags is unlikely to provide significant model improvements, particularly for the short lead times tested here. Instead, regressing on a seasonal variable should allow exploitation of these slight seasonal variations.

Taking these cross-correlation and temporal correlation results together, there may be some time-delayed cross correlation that could be captured by multisite models (that use past information from many nearby or distant sites). Nonetheless spatial correlation at time lag zero dominates over temporal correlation at any time lag.

4. Methods

This section details spatiotemporal statistical models of rainfall totals at the 10 selected Thames Valley locations, and the forecast performance comparisons of these models against unsophisticated benchmarks and postprocessed ensemble NWP.

b. Comparing daily rainfall point forecasts

Here we compare the ability of the models to produce point forecasts of daily total rainfall amount, which for the appropriate models is the combined model of occurrence/intensity. The point forecast from each model is the median of the model’s forecast density. The mean absolute error (MAE) score is used:

View Expanded

where x̂_n^m is a forecast of total rainfall amount and L is the test data length. This score is proper (Gneiting and Raftery 2007) meaning that lower MAE scores imply more accurate forecasts, and the score is minimized by the perfect forecast.

c. Comparing daily rainfall density forecasts

The assessment of density forecasts is somewhat more complex than point forecasts. Specifically, it is important that the forecast produces the correct density of the observations: it must be well calibrated, and at the same time maximize sharpness; each forecast density must have a high probability around the actual observations (Diebold et al. 1998; Gneiting et al. 2007). Here we use the continuous ranked probability score (CRPS; Gneiting and Raftery 2007) which is also proper, and it can be shown decomposable into separate components of both calibration and sharpness. Thus, small values indicate forecasts that are both well calibrated and sharp. We use the empirical form (Gneiting and Raftery 2007):

View Expanded

where X and X′ are independent random variables drawn from model’s forecast density function p, and E denotes expectation.

We also perform diagnostic checks of the forecast calibration using the probability integral transform (Diebold et al. 1998; Gneiting et al. 2007):

View Expanded

(for notational clarity we have x = x_n). Here the function p is the (unconditional or conditional) forecast density function (or transition function) of each of the models, at forecast lead time of 1 day. For the perfectly calibrated model, z_n will be i.i.d. with uniform density in the interval between 0 and 1. Therefore, measuring calibration requires assessing the extent of deviation from uniformity of this time series. Typically, if the histogram is “U shaped” it will be because the spread of the forecasts is too narrow. Conversely, a humped-shaped histogram will indicate overdispersed forecasts (i.e., their range is too large; Gneiting et al. 2007).

Similarly, if the model captures the serial dependence in the time series, then z_n will be serially independent. Tests for serial independence using the autocorrelation are most often applied in this context, and we follow this practice here (Gneiting et al. 2007), displaying the standard Bartlett 95% autocorrelation confidence intervals. We employ the stochastic interpolation method to calculate the PIT, by drawing 1000 samples from each predicted density and constructing the empirical cumulative density function. This defines a discrete distribution that approximates the underlying mixed discrete continuous density function (see Smith 1985 for further details).

d. Comparing daily rainfall occurrence probability forecasts

We compare probability forecasts of occurrence using the Brier score, which is also a proper score (Brier and Allen 1951):

View Expanded

where q̂_n^m is the forecast probability of occurrence and L is the test data length.

5. Results

For point forecasting performance MAE of rainfall totals, Fig. 8 shows that at lead times of 1–8 days ahead, the multisite nonseasonal/seasonal Markov SMGLM ranks slightly better than the processed ensembles. At lead times of between 9 and 25 days ahead, the Markov MGLM is best. The postprocessed ensembles have skill over climatology at the available lead times. The i.i.d. gamma–Bernoulli model does not have skill at any forecast horizon. The Markov GLM is an improvement over the climatology for the first day, but afterward, ceases to have skill. The exponential mixture model only has skill on the first day, and thereafter loses skill. The persistence forecast is consistently the worst forecast over all horizons.

Turning to the density forecasts, the diagnostic checks in Fig. 9 show that all of the models are reasonably well calibrated, although there is some residual over- and underdispersion in most models. From the autocorrelation functions of the z series, as expected the unconditional climatology and i.i.d. gamma–Bernoulli models fail to capture the small amount of serial correlation in the data for the first 4 or 5 days time lag. The conditional models naturally fare better in this regard, and the postprocessed ensembles perform best. The results are consistent with these findings for the other gauges, with some negligible differences. The diagnostic check of Fig. 5 shows that the Markov JGLM is capable of reproducing the spatial correlations to a reasonable extent, although the correlations are smaller than those in the original time series because the mixed GLM density model of the actual probability densities of rainfall at each site is not perfect.

Regarding combined calibration and sharpness of the density forecasts, Fig. 10 shows that the nonseasonal/seasonal multisite Markov GLMs are the best performers on the first day. The joint-site GLM has some skill at 1 day ahead, but thereafter lacks appreciative skill. At 2–8 days ahead, the postprocessed ensembles rank first, just ahead of the Markov MGLM/SMGLMs, which have some skill for some forecast horizons between 9 and 25 days ahead.

The i.i.d. gamma–Bernoulli model does not have skill over climatology at any horizon. The rest of the conditional models show slight improvements at 1 day ahead, but then show similar performance to the unconditional models.

For the occurrence Brier score, the postprocessed ensembles have the best score for forecast horizons of 2–8 days, outperforming all other models. However, the conditional models, in particular the Markov MGLM/SMGLMs all show skill relative to climatology for the first day, thereafter they lose skill. The i.i.d. gamma–Bernoulli model does not have skill at any forecast horizon, and the persistence forecast is the worst at every horizon.

It is worth noting that Markov GLMs involve highly nonlinear feedback mechanisms, particularly noticeable when propagating information from many neighboring sites. Although often drifting to zero, unlike the simple, unconditionally stable Markov chains such as the Cao–Li–Wei model, it is possible for Markov GLMs to produce growing responses as well. This is noticeable in the MAE, CRPS, and Brier scores, where the performance of some of the more complex Markov models varies somewhat with forecast horizon. The other, simpler models produce smoother results. Another note is that experiments with shuffling the order of sites used in the Markov JGLM method did not lead to substantial differences in performance, either in the MAE, CRPS, or Brier scores.

These forecasting results raise the question of why some of the statistical methods have comparable or better performance than the postprocessed ensembles in some aspects as previously described. Turning first to point forecast performance, temporal correlations in the data beyond 1 day ahead are very small. Nonetheless, the Markov GLMs are well equipped to exploit this small, 1-day-ahead autocorrelation.

Second, with regard to density forecasts, the CRPS results show that incorporating all the information from every site on the previous day when forecasting each site individually, improves the calibration of the Markov multisite models relative to all the other models (including the Markov JGLM joint site method). Therefore, neighboring sites do contain useful information that can be exploited to improve forecasts at short lead times.

Regarding the occurrence forecast performance, the ensemble calibration regression method is successfully able to remove bias in the interpolated ensembles to produce the best forecasts. However, the training data is very similar to the test data, both having long, consecutive runs of dry days, followed by shorter, consecutive runs of wet days. Thus, the occurrence time series is highly autocorrelated 1 day ahead, diminishing rapidly with increasing forecast horizon. The conditional Markov MGLM and Markov SMGLMs use the most information from the past in order to produce forecasts. As can be seen in Fig. 11, these conditional models, which are designed to capture temporal autocorrelation, do very well in exploiting this 1-day-ahead autocorrelation.

6. Conclusions

In this paper, we investigated the autocorrelation and cross-correlation structure of a large number of rain gauges in the Thames Valley, United Kingdom, and demonstrated that while autocorrelation in the rainfall depth amount is of minor importance, spatial cross correlation is highly dominant. We also showed some slight seasonal variations in the mean intensity of rainfall on wet days. We used this information to produce a set of new, site-specific statistical density forecast models in this spatial area, based on variations of a Markov GLM, non-Gaussian regression method in a couple of different configurations. We tested these models against a set of simple benchmarks and some more sophisticated models proposed in the literature, and against ensemble NWP forecasts combined with GLM regression in a postprocessing approach. The tests involved the comparison of rainfall forecast performance of all the models for each rain gauge, up to 45 days ahead. The tests demonstrated that Markov GLMs can be configured to produce good 1-day-ahead forecasts and reasonably skillful short-term forecasts up to a couple of weeks ahead. They also show that combining GLM regression with ensembles can effectively calibrate the ensembles to produce skilful density forecasts up to a week ahead. The results do not support the use of any of the other proposed models.

In terms of overall density forecasting, all the models were well calibrated, but in summary, the GLMs, either alone or in combination with ensembles, performed best when both calibration and sharpness were considered simultaneously. In terms of ability to forecast occurrence of rain, except at lead times of 1 day, no models were capable of bettering the postprocessed ensembles. We also demonstrated the superior ability of the postprocessed ensembles to reproduce the (small) serial correlation in the rainfall data.

A similar study (Taylor and Buizza 2004) compared temperature forecasts from simple autoregressive time series models, postprocessed European Centre for Medium-Range Weather Forecasts (ECMWF) ensemble mean, and a high-resolution point NWP; it was found that the ensemble mean was the best under the MAE at up to 10 days ahead. Similarly, Campbell and Diebold (2005) found that point forecasts from time series models could not outperform NWP forecasts. Our findings disagree as we have found it possible to produce time series point forecasts slightly better than calibrated NWP forecasts up to 8 days ahead. We believe this is because precipitation is notoriously difficult to predict, particularly at local sites, and time series models exploiting correlations can contribute to making useful forecasts.

The results lead us to suggest ways in which ensemble forecasts might best be calibrated for full-density forecast applications. Contrary to other reports (Cao et al. 2004; Robertson et al. 2004), we believe that this study can act as a caution against the use of simple unconditional density models and the more elaborate two-state Markov chains combined with exponential mixtures for this purpose. In particular, we believe our results suggest that Markov GLMs could be effective new techniques in this regard, which concurs with other studies (Sloughter et al. 2007).