## 1. Introduction

Statistical downscaling is a class of methods used for modeling the impact of regional climate variations and change on daily rainfall at local scale, for example, in agricultural applications of climate forecasts (e.g., Hansen et al. 2006). Hidden Markov models (HMMs) have been applied quite extensively to simulate daily rainfall variability across multiple weather stations, based on rain gauge observations and exogenous meteorological variables (Hay et al. 1991; Hughes and Guttorp 1994a; Charles et al. 1999; Bellone et al. 2000; Robertson et al. 2004; Greene et al. 2008). In these multisite stochastic weather generators based on discrete-state HMMs, each day is assumed to be associated with one of a finite number of hidden states, where the distributional characteristics of the states are estimated from historical data. The state-based nature of the HMM is well suited to representing large-scale weather control on the local rainfall processes, where the control is manifested across a region and influences individual locations according to local surface conditions such as topography and land use. An important goal of climate downscaling research is to better understand this cross-scale linkage, in order to obtain estimates of climate variability and change at local scale that better represent the physical relationships between large and small scales.

In a nonhomogeneous HMM (NHMM), the state transition probabilities are conditioned by one or more exogenous input variables. This formulation combines the Markov chain, to model the weather element as a stochastic process, with the influence of large-scale exogenous meteorological or climatic variables, such as spatially averaged geopotential height fields (Hughes et al. 1999) or general circulation model (GCM) output (Robertson et al. 2004, 2006, 2009). However, the NHMM presents a limitation for downscaling of climate change simulations because the rainfall characteristics of the modeled states may evolve as the climate warms (Timbal et al. 2008; Greene et al. 2011).

Furrer and Katz (2007), Ambrosino et al. (2014), and Kenabatho et al. (2012) have taken an alternative approach to stochastic weather generation, based on a generalized linear model (GLM) that allows state-dependent model parameters to be influenced directly by exogenous variables such as seasonal effects or climate indices. The GLM framework can be viewed as a generalization of classic least squares linear regression modeling (see McCullagh and Nelder 1989), allowing the conditional mean of a variable of interest to be modeled as a function of predictor variables, with noise characteristics that are non-Gaussian, the latter feature being particularly useful for modeling precipitation data.

In this paper we take a combined GLM–HMM approach that uses a GLM to incorporate the impact of exogenous variables (here seasonality and interannual regional climate variability), while exploiting an HMM for the stochastic spatiotemporal variability of daily rainfall across a regional network of stations. Thus, rather than encode the climatic influence on weather in terms of a set of fixed states as in an NHMM, the characteristics of the rainfall states themselves are assumed to be modulated in time by climate. In this way, the model emphasizes the regression between local rainfall parameters and the exogenous controls. The states enable more realistic modeling of spatiotemporal structures than would a stateless model, while allowing for a hierarchical structure of the climatic influence that can, for instance, be made to act on all stations equally. A discussion of the pros and cons of this approach is presented in section 6.

Two contrasting monsoonal regions, each characterized by increasing societal water demand, are taken here as case studies: the Punjab region of northern India and Pakistan and the upper Yangtze River basin in south-central China. The southwest monsoon over northern India is characterized by intense rainfall over a relatively short (July–August) peak season, with pronounced intraseasonal active and break phases and year-to-year variability (Greene et al. 2008). The summer monsoon over western China lasts longer (May–August peak season) with less prominent seasonality. These two regions provide much of the inflow to the Indus River in Pakistan and India and the Yangtze River above the Three Gorges Dam in China, respectively, and are thus of great importance for water resources, agriculture, and hydropower generation for these countries (Immerzeel et al. 2010). For each region, we evaluate the model in terms of its ability to represent seasonality of rainfall and the marginal (i.e., climatological) distributions of daily rainfall. We also evaluate its ability to capture interannual variability at the station level in daily rainfall characteristics, including rainfall frequency and extremes, when forced with regional-scale spatiotemporal averages of observed rainfall as the exogenous variable, and use the HMM’s states to interpret the multiscale nature of the rainfall variability in the two regions.

The paper proceeds as follows. The new model, which combines GLM and HMM approaches, is described in section 2, and the rainfall and other meteorological datasets are described in section 3. In section 4, the inferred rainfall states for Punjab and the upper Yangtze are presented and their meteorological and climatic associations are analyzed. In section 5, we then examine rainfall simulations generated by the model. Conclusions are presented in section 6.

## 2. Using GLMs with HMMs for precipitation modeling

Let *t* at station or site *s*. The index *t* refers to both the day of year as well as to the year itself. We will model

*s*when the hidden state variable

*t*is in state

*k*. Two key assumptions underlie this model. First, the hidden states are assumed to be first-order Markov, that is,

*K*×

*K*state-to-state transition matrix

*t*,

*t*. Both assumptions lead to the familiar product form for the joint distribution of observations and states aswhere

In precipitation modeling applications, it is common to make an additional assumption that precipitation amounts at individual sites on day *t* *K* discrete weather states ensures that some degree of spatial dependence across sites is captured via the latent states. Indeed, this ability to capture spatial dependence in a relatively parsimonious manner is one of the features that makes HMMs attractive for multisite precipitation simulation at daily time scales. The assumption that precipitation amounts are “locally” (in time) independent given the weather state

The extension of the homogeneous HMM to the nonhomogeneous case, by making the transition probabilities in

As an alternative, for the single-site case, Furrer and Katz (2007) proposed a GLM approach that allows the distributional characteristics of daily precipitation to be influenced directly by exogenous variables such as seasonal effects or climate indices. In the approach of Furrer and Katz, daily precipitation intensity at an individual site was modeled as having a gamma distribution with mean *μ* being an intercept term, and the

*z*, and exogenous variables

*s*on day

*t*, conditioned on the value of the Markov hidden state variable

are the probabilities (mixing weights) for the three mixture components, with , where (as described below) the *ϕ*values in turn are functions of the exogenous variable; is the first mixture component, a delta function at representing zero precipitation; - and
are two gamma densities representing mixture components for light ( *j*= 1) and heavy (*j*= 2) rainfall amounts, with parametersand that depend on the component *j*and (for*λ*) on the state valueand site *s.*

In this paper we used fixed values for the *α* parameters, specifically *α* parameters chosen to allow for various shapes of the distribution) and found that the choice of one exponential and one gamma component (as used in this paper) generally produced the best results (lowest BIC scores).

BIC scores for models with different numbers of mixture distributions. A min BIC score (in boldface) is related to the best model. The first entry (*α*) of the gamma distribution is fixed and the second one (*λ*) is variable for each of the mixture distributions. When the first parameter is set to 1, it is equivalent to an exponential distribution.

Other parameterizations for the mixture components could also be used in principle within the GLM–HMM framework, such as extreme value distributions (e.g., Vrac and Naveau 2007; Johnson et al. 1996).

We allow the mixing weights *s* to change in response to external influences. In particular, let *s* on day *t*. (In practice the exogenous input variables may not be available at a daily scale, but instead at a monthly or annual time scale.) We use a GLM approach to link a linear function of the exogenous variables, *s* and each state *k*.

*s*at time

*t*. The multinomial probit model operates as follows: with

*J*= 3 mixture components (and mixture probabilities), we define

*J*+ 1 = 4 bin boundaries

*h*= 0, 1, 2, 3), where

*t*is in state

*k*. The mixture probabilities

*j*= 0), light rain (

*j*= 1), and heavy rain (

*j*= 2)].

Figure 2 provides a simple illustration of the approach. Figure 2 (top left) shows the normal distribution

We also allow the transition matrix for the hidden states *K* × *K* matrix of transitions, while being short enough to resolve the strong seasonality (especially for Punjab) within the short June–September (JJAS) season.

Letting

## 3. Rainfall data

Figure 3 shows the weather station locations for the Punjab and upper Yangtze River regions, situated between about 25° and 35°N. Rainfall station data were obtained from the National Centers for Environmental Prediction (NCEP) Climate Prediction Center (CPC) Global Summary of the Day (GSOD) observations for 1980–2010 (CPC/NCEP 1987). We selected the 22 stations in the Punjab region and the 52 over the upper Yangtze that have fewer than 20% missing daily readings for the JJAS season.

The JJAS season was selected to approximately span the summer monsoon in both cases, accounting for the greater part of annual precipitation totals (73% in the Punjab region and 64% in Yangtze). The stations are from approximately 50 to a few hundred kilometers apart, motivating the structure of the GLM–HMM in which station rainfall is modeled as spatially independent, given the large-scale weather state.

Figure 4 shows spatial summaries of empirical statistics for both regions, including mean daily rainfall (Figs. 4a,d), probability of rain (Figs. 4b,e), and mean daily intensity (Figs. 4c,f). Mean daily intensity is defined as the average rainfall only for days it rained (≥0.1 mm day^{−1}), while mean daily rainfall includes all days. Over the Punjab, rainfall probabilities are highest in north-central India, while mean daily intensities are largest in northern Pakistan; the desert region to the west experiences the lowest rainfall amounts. The upper Yangtze exhibits comparable station amounts to the Punjab, with larger values toward the north and east where mean intensities are largest. Rainfall probabilities over the upper Yangtze, by contrast, are larger in the western region. Mean JJAS station rainfall ranges from 0.5 to 6.5 mm day^{−1} (station average 3.0 mm day^{−1}) for the Punjab region and from 2.1 to 8.8 mm day^{−1} (station average 4.8 mm day^{−1}) for the Yangtze.

### Exogenous variables

We use a two-dimensional exogenous variable *s*.

The second of the two exogenous variables in

The SAI is not highly correlated (cor) with the Niño-3.4 sea surface temperature index of ENSO for either region [cor(SAI-Punjab, Niño-3.4) = 0.24, cor(SAI-Yangtze, Niño-3.4) = 0.14], and the SAI variables for the two regions are relatively uncorrelated [cor(SAI-Punjab, SAI-Yangtze) = 0.10]. The variance (var) of the SAI provides a measure of the spatial coherence of the rainfall field, ranging from var(SAI) = 0 for an uncorrelated field to var(SAI) = 1 for a perfectly correlated field (Moron et al. 2007). The values for the two networks of stations are var(SAI-Punjab) = 0.30 and var(SAI-Yangtze) = 0.17, indicating low spatial coherence of interannual rainfall variation in both cases compared to those found previously for other networks of rainfall stations (Moron et al. 2007). These values indicate intrinsic limitations to downscaling of interannual rainfall variability from these regional aggregates, particularly for the larger Yangtze network.

## 4. Inferred states and parameters

### a. Choosing the number of states

To fit the GLM–HMM, we first selected the number of hidden states for each of the two regions. We fit the model with different numbers of states *K* that minimized the BIC. We also experimented with the deviance information criterion (DIC; Spiegelhalter et al. 2002) for model selection but found that it consistently preferred the models with the largest numbers of parameters—this may be because of nonnormality in the parameter posterior distributions, a known issue for DIC. For this reason we opted to use BIC rather than DIC in this work. BIC is defined (for a given *K*) as the negative log likelihood of the data plus a penalty term for model complexity (Schwarz 1978). We use the approach of Scott (2002), adapted to our model, to compute the log likelihood in a computationally efficient and stable method using a recursive algorithm. The GLM–HMM for the Punjab region has a minimum BIC score with *K* = 4. For the Yangtze basin region, the BIC is minimized for *K* = 7, but the BIC values for *K* = 6, *K* = 7, and *K* = 8 are not significantly different, so we selected *K* = 6 for parsimony. The larger number of states in the Yangtze model is consistent with the larger number of Yangtze stations and their smaller spatial coherence with respect to interannual anomalies compared to the Punjab.

In practical applications of model selection, it is common to augment quantitative model selection criteria (such as BIC) with additional assessment of other attributes of the model (e.g., Shirley et al. 2010). In this spirit, we use the BIC to inform our decision but also investigate (below) various aspects of the model and its interpretation.

### b. Estimated GLM parameters

As described in the appendix, we use a Bayesian approach to generate samples of the unknown parameters and the hidden states from their posterior distributions conditioned on the data. Each iteration of the sampling algorithm generates a single sample for each of the parameters and states. In the results in this paper, we used *N* = 2000 iterations (or samples).

Figure 6 shows the mean estimated coefficient values for each station and their 95% probability intervals for the seasonality and SAI inputs. The mean values and 95% intervals are computed from the *N* = 2000 parameter samples. With one exception, the probability intervals do not include zero (indicated as a horizontal line), indicating that the *β* coefficients associated with the exogenous inputs of both seasonality and interannual variability are statistically significant from a Bayesian perspective.

### c. Punjab rainfall states

The rainfall states encode the seasonality of station-scale rainfall, as well as the patterns of subseasonal weather variability. After running the sampling algorithm on the GLM–HMM with 30 years of data, for each day *t* in the data we have *N* = 2000 different samples of the hidden states *t* to the most frequently occurring state for that day across the *N* samples, providing state assignments in a manner similar to that computed via the Viterbi algorithm (Forney 1973) under a maximum likelihood framework as used in previous rainfall modeling studies with HMMs (e.g., Robertson et al. 2004). This maximum a posteriori probability (MAP) estimate of the state sequence is used to gain an interpretation of the states as an important biproduct of the HMM, while the full posterior draws are used in analyzing the rainfall simulations in section 5 below.

Figure 7 shows the probability of rainfall (Figs. 7a–d) and mean daily intensity (Figs. 7e–h) for each of the four states in the model for the Punjab region. The states are loosely ordered from wettest to driest. As found in Greene et al. (2008), who used a four-state HMM for rainfall over a similar region, there is a state that is generally wet at all stations (state 1) and one that is dry at all stations (state 4), together with two intermediate states with a somewhat north–south gradient. There is a general correspondence between high rainfall probability and high mean intensity for the wetter states, as found in Greene et al. (2008).

Figure 8 shows the estimated state sequence, together with its seasonality averaged across the 30 years. Again, each day *t* is assigned to the state that occurs most frequently for that day across the *N* samples. The dry state occurs more frequently in June and September at the expense of the wetter states, though not exclusively so, and there are rapid fluctuations in time between the states. These rapid fluctuations are more pronounced than for a similar HMM analysis of northern India by Greene et al. (2008), and seasonality is less prominent. This is consistent with the greater aridity of the Punjab region. As seen in Greene et al. (2008), extended monsoon-break episodes of the dry state occur in some years. There is no obvious trend in the year-to-year occurrence of the states.

### d. Upper Yangtze basin rainfall states

Figure 9 shows the same types of plots as in Fig. 7 for the upper Yangtze basin, showing the probability of rainfall (Figs. 9a–f) and mean daily intensity (Figs. 9g–l) for each of the *K* = 6 states. The states are again loosely ordered from heavy to light rain. As with the Punjab region, there is a general correspondence between high rainfall probability and high mean intensity for the wetter states. Compared to the spatial variation in the states for the Punjab region (Fig. 7), there is considerably more spatial variation within each state over the larger Yangtze basin network. State 3 is characterized by high rainfall probabilities over the western part of the domain with low mean intensities indicative of drizzle, for example.

Figure 10 shows the estimated state sequence and its seasonality for the Yangtze basin network, analogous to Fig. 8. The sequence is much noisier than for Punjab and average seasonality is less easy to discern. This is partly a function of the total number of states (six vs four for Punjab) and illustrates the larger complexity of the region’s subseasonal rainfall variability. The monsoon in the Yangtze begins earlier in the calendar year than the Punjab region and is more homogenous throughout the season. There is again no obvious trend in the year-to-year occurrence frequencies of the states.

## 5. Rainfall simulations

### a. Seasonality

In Fig. 11, we assess the quality of the model in terms of reproducing the seasonal cycle. The black line shows the seasonal cycle of the observed and mean simulated rainfall amount for the Punjab (Fig. 11a) and the Yangtze basin (Fig. 11b), obtained by averaging rainfall amount over each station for each calendar day over 31 years. The mean simulated amount per day and 95% confidence bands are also shown in gray, as computed from 500 simulated 31-yr datasets (as described in the appendix) conditioned on the observed SAI and seasonal probability of occurrence as exogenous variables. The 500 × 31 simulated daily time series are then averaged over each calendar day (and over stations) to produce the mean and probability bands in Fig. 11. The model captures the seasonality of rainfall amount in both regions, with the simpler monsoon seasonality in the Punjab region more accurately represented. This may be because of greater spatial heterogeneity in seasonality across the larger Yangtze region where some stations have early seasonal peaks, while others may be multimodal or have seasonal peaks later in the season (not shown).

### b. Comparisons of distributions and extremes

#### 1) Modeling the rainfall distribution

Figure 12 shows the observed rainfall distributions for selected wet and dry stations from each region (results for other stations were found to be generally similar). The observed log frequencies of rainfall (histogram) are compared to 500 simulated datasets of 30 years of data generated via conditional simulation from the model, represented by 95% confidence intervals plotted on the histogram. The model is seen to capture the distributions well in both regions, for both the wetter and drier stations. There is a slight bias in the Yangtze heavy rain station between 30 and 50 mm, where the model is overestimating days in this range, but otherwise the emission distributions are generally a good fit to the observed distributional nature of the data.

#### 2) Extreme rainfall events

We investigated the model’s ability to capture extreme behavior in terms of daily amounts. The model was not specifically designed to model extreme events and relies solely on the tail behavior of the gamma distribution to model the extremes (we are not suggesting to replace extreme value theory models when the extremes are the primary interest, but rather here we evaluate one of our model’s attributes, which is its ability to capture extreme behavior). The estimation of the parameters of the gamma distribution tends to be dominated by the more frequent low rainfall values rather than the much rarer extremes. We fit a generalized extreme value (GEV) model to the annual maximum rainfall at each station, using the extremes library in R, and report the return levels of 50-yr return period events (Furrer and Katz 2007). Figure 13 shows summaries of 500 simulations from the model as box plots, one per station. Despite its simplicity, the model captures the extremes relatively well in the Punjab region, although it tends to underestimate them in the upper Yangtze basin.

### c. Interannual variability

As mentioned above, the exogenous variables in the model include 1) a station-scale, seasonal-cycle component and 2) a regional-scale, seasonal-average variable, the SAI. In this section, we evaluate the interannual predictive skill of the model in response to the SAI variable. In the results below, models are built both with and without the SAI variable (for comparison purposes), and all models include the seasonal-cycle exogenous variable. The evaluation is carried out via sixfold cross validation, where six different consecutive 5-yr blocks of data are held out in turn (except for the year 1980), and the model is fit to the other 26 years of data. For each fitted model we generate 1000 predictive runs for each of the 5 held-out years, conditioned on the SAI value for that year (the appendix provides a more detailed description of how predictive runs are generated).

Figure 14 shows the results of these conditional predictions for the Punjab (Figs. 14a,c,e,g) and Yangtze (Figs. 14b,d,f,h). Various predictive statistics of interest (*y* axis) are shown per year (*x* axis) from 1981 onward, where all statistics are computed by spatially averaging the simulated rainfall across all stations. Figures 14a and 14b are shown for calibration purposes and show the annual-average rainfall when predictive simulations are generated without using the SAI input variable. The boxplots reflect the variability (from the model) over the 1000 simulated years, where the median (50% percentile) is shown as a horizontal black line and the 25th and 75th percentile are the lower and upper bounds of the shaded box, respectively. The black dots show the actual observed average rainfall per year. The observed rainfall varies significantly from year to year, while the model simulations (without any interannual input) remain relatively constant in their distribution (as one would expect, since any variation from year to year is entirely due to sampling variability in the simulations).

Figures 14c–h show the same type of plots, now with the SAI variable included in the model, for three specific metrics of interest (Hughes et al. 1999; Joshi and Rajeevan 2006): mean daily rainfall (averaged over the JJAS season), the count of rainfall days over 70 mm (0.6% of Punjab days and 0.5% of Yangtze days), and the count of dry days (77% of Punjab days and 45% of Yangtze days). For the average rainfall metric (Figs. 14c,d) the model’s simulations fairly accurately bracket the observed rainfall for both the Punjab and the Yangtze, which is anticipated given that the input is the SAI. The model also performs fairly well for the annual number of simulated dry days (Figs. 14g,h), for both regions. For the 70-mm rainfall extremes (Figs. 14e,f), the model captures some of the observed interannual variation over the Punjab but not for the Yangtze.

Figure 15 shows results from the same simulations as scatterplots. Each point in each plot represents the root-mean-square error (RMSE) for a particular year (again for the rainfall averaged over stations), where for each such year the RMSE is computed between the 1000 simulated years and observed statistic for that year. The dots are shaded, with darker shading corresponding to wetter years (according to the respective statistic), and the RMSE values averaged over all years are given in the top left of each panel. The *x* axis corresponds to not using the SAI variables as an exogenous input and the *y* axis corresponds to including SAI. In the case of dry-day counts (Figs. 15e,f), including the SAI leads to a significant reduction in RMSE for most years (i.e., many of the yearly points are well below the diagonal), particularly over Punjab. This occurs to a lesser extent for seasonal rainfall amount, and only marginally for heavy rain days, and then for only 6 years over Punjab (cf. Fig. 14). There is no clear correspondence between the size of the RMSE and wetter or dryer years (dot shading).

Figure 16 shows analogous RMSE plots, but where each point now represents an individual station and the averaging is performed over years to show the impact of the SAI at each location, thus depicting the model’s downscaling skill. For each station, the rainfall statistic is first computed for each year of the 1000 simulations and the observations; the ensemble mean of the simulations is then computed, and the square of the differences is computed for each year. The mean over years is then taken and the square root plotted in Fig. 16. Again, the largest impact of the SAI is seen for dry-day count, and for the Punjab region, with less impact of the other variables, especially 70-mm rain days.

In summary, Figs. 14–16 demonstrate that the GLM–HMM with SAI of seasonal regionally averaged rainfall does exhibit skill (in terms of reduced RMSE) at the station level and for individual years, but that this skill is largely limited to dry-day counts, though with some skill for seasonal rainfall total over the Punjab. These findings are discussed further in the conclusions section.

### d. Comparison to the NHMM

In this subsection we compare the model’s performance with that of an equivalent NHMM, similar to those used by Hughes et al. (1999) and Robertson et al. (2006). Here the NHMM estimation is Bayesian as in the current case, but with the same two-dimensional exogenous input used to modulate the NHMM’s transition probabilities. Here, we use the station-average seasonality in place of the station-scale one used in the GLM. The same number of states were used in each case.

Figure 17 shows the NHMM’s simulation of the seasonality (Figs. 17a,b), and interannual performance of dry-day counts (Figs. 17c,d). The seasonal cycle for the GLM–HMM (Fig. 11) and the NHMM (Figs. 17a,b) are quite similar for the Punjab region, but the GLM–HMM clearly outperforms the NHMM over the Yangtze basin. The interannual performance of the two models is very close, with the RMSE of dry-day counts of the two models plotted against each other stationwise in Figs. 17c and 17d. The station averages of the RMSEs are also given in the caption of Fig. 17 for dry-day counts, average rainfall, and number of 70+ mm days. These again show very similar numbers for the two models, though the GLM–HMM has a slight edge (smaller RMSE) overall.

Table 2 shows the Markov conditional transition probabilities of dry–dry, dry–wet, wet–dry, and wet–wet day transitions at the station level for the NHMM and GLM–HMM. The first row for each region gives the probabilities from the observed data (missing observations are not included). Both the NHMM and GLM–NHMM capture these transition probabilities reasonably well. Values for the GLM–NHMM are marginally closer to observed than those from the NHMM, although differences are small.

Markov conditional rainfall occurrence probabilities pooled over stations for the observed data and for two models for Punjab and the Yangtze basin.

### e. Meteorological associations with rainfall states

The HMM is designed to represent synoptic-scale subseasonal rainfall variability and seasonality across the rainfall network, with interannual variability and additional seasonality encoded via the GLM. To gain insight into the synoptic-scale rainfall patterns associated with the states, we plot composites of horizontal fluxes of moisture and vertical air motions, averaged over the days assigned to each state. These diagnostics are calculated based on NCEP–NCAR (version 1) reanalyses data (Kalnay et al. 1996) and presented in terms of deviations (anomalies) from the JJAS climatological time averages displayed in Fig. 18. The moisture flux is integrated vertically from the surface to 200 hPa. This seasonal-average climatology exhibits a broad moist monsoon southwesterly flux with maxima over the Arabian Sea and Bay of Bengal, located upstream and to the south of the Punjab and Yangtze regions, respectively, together with a weaker southerly flux from the South China Sea into China. A region of maximum climatological ascent at 500 hPa (negative omega) is located over the Bay of Bengal, with our rainfall networks situated on its northwest and northeast margins.

Figure 19 shows the composite atmospheric anomalies for the Punjab states. State 1 (wettest) shows an enhanced southwesterly moisture flux from the Arabian Sea with a cyclonic anomaly to the north and strong convergence and anomalous ascent over Punjab. The main moisture source is over the Arabian Sea, even though the path of moisture advection crosses the northern Bay of Bengal. State 1 is most frequent from late June to mid-September (Fig. 8) and is representative of active monsoon conditions; Fig. 19a can thus be interpreted as the difference between active monsoon conditions and the JJAS seasonal average, reflecting both seasonality as well as subseasonal phenomena (Greene et al. 2008). State 3 exhibits a rather similar pattern, though weaker, and with less extension to the north, consistent with the more southerly location of the largest rainfall probabilities in Fig. 7. State 2 presents a different pattern indicating a northward displacement of the mean moisture current, while the dry state 4 is almost a mirror image of state 1, with an anticyclonic circulation anomaly. The emergent mechanistic picture is that of a monsoon trough anomaly—either strong (state 1) or weak (state 3)—or a northward displacement of the mean moisture current toward the Himalayan foothills (state 2), consistent with the HMM of Greene et al. (2008) over a region slightly farther south.

Figure 20 shows composite atmospheric anomalies for the Yangtze states. State 1, the wettest, is characterized by a northward displacement of the westerly monsoonal moisture current near 25°N and broad anomalous ascent over the Yangtze basin. States 2–4 are characterized by regionally limited Rossby-wave-like circulation anomalies. States 2 and 4 exhibit an anticyclonic wave centered over eastern China with a southerly flux of moisture on its westward flank, in both cases with dipolar vertical motion anomalies and ascent on the northern flank. State 3 is a “drizzle” state with very small rainfall intensity, small ascent yet relatively large northerly moisture fluxes. State 5 is relatively wet in the south, associated with southerly moisture flux. In summary, the state rainfall distributions in Fig. 9 are consistent with the composite moisture flux and vertical velocity anomalies, while Rossby waves play a larger role in explaining the rainfall anomalies in this region, consistent with its location downstream of the Tibetan plateau. Seasonality is weaker and the HMM component of the GLM–HMM thus largely represents this Rossby-wave-like subseasonal variability.

## 6. Summary and conclusions

We have presented a new approach to multisite precipitation downscaling that differs from previous nonhomogeneous HMMs through its incorporation of exogenous variables as direct modifiers of the mixture of the emission distribution at each station, rather than using logistic regression to influence the transition probabilities of the Markov chain. The new model combines a weather state model (via an HMM) to capture spatial station dependencies, and generalized linear model (GLM) to incorporate exogenous variables at a station level (independent of the HMM state) to downscale precipitation. Parameter estimation for the model is performed using a Bayesian framework, allowing (for example) the ability to average over parameter uncertainty when generating simulations from the model.

Two illustrative applications of the model were presented for regions of monsoonal Asia, the Punjab and upper Yangtze River basin. For each region we used two exogenous variables as input: the seasonal-cycle-dependent likelihood of precipitation occurrence at each station, and a standardized seasonal-average rainfall amount averaged over all stations (the SAI). The first variable provides a simple way of incorporating seasonality at the station level, while the second allows us to test the model’s ability to disaggregate or “downscale” seasonally and regionally averaged rainfall amounts to daily sequences at the station level. These choices are meant as simple illustrations, and the predictors could be chosen separately for each station, and vary daily if desired.

The model performance is evaluated using a large ensemble of cross-validated daily rainfall simulations, in terms of seasonality, daily rainfall distributions, 50-yr return levels, and interannual variability of daily rainfall characteristics. The model captures the marginal daily distributions well in both regions, for both the wetter and drier stations (Fig. 12). Despite its simplicity, the model captures the extremes relatively well in the Punjab region, although it tends to underestimate them in the upper Yangtze basin (Fig. 13). In terms of interannual variability, the GLM–HMM with SAI of seasonally and regionally averaged rainfall as a predictor is shown to exhibit skill (in terms of reduced RMSE) at the station level and for individual years at the regional level, particularly for the Punjab region. The skill is largest for dry-day counts, with less skill in the seasonal rainfall totals, and almost none in the number of extreme wet days (Figs. 14–16).

The model’s GLM component relates the exogenous variables to each station’s rainfall distribution expressed as a mixture of dry-day, light rain, and heavy rain component distributions via a mixture model (Figs. 1, 6). Previous studies have found dry-day counts to be more seasonally predictable than seasonal rainfall totals in monsoonal climates (Robertson et al. 2009), and this is confirmed by our results (Figs. 14–16) for the two summer monsoonal regions considered here, in the sense that interannual variations of dry-day counts at local scale are more closely tied to the regional SAI of seasonal total rainfall than are the station totals themselves (Fig. 16).

The higher model skill in downscaling of rainfall frequency at local scale compared to seasonal rainfall total can be interpreted via the higher spatial coherence of seasonal anomalies of rainfall frequency compared to mean daily intensity (Moron et al. 2007). The spatial coherence given by the variance of the SAI computed from the Punjab network is 0.30, 0.47, and 0.10 for seasonal rainfall total, dry-day count, and 70-mm wet-day count, respectively. The numbers for the Yangtze network are 0.17, 0.38, and 0.09. Both sets of numbers are consistent with the RMSE results in Figs. 15 and 16, highlighting that the spatial coherence of the field dictates the ability of the model to downscale rainfall from the regional average.

The HMM’s states represent spatial patterns of regional-scale rainfall variability (Figs. 7a, 9a) inferred from the daily rainfall data, while the estimated state sequence shows their temporal evolution over the historical period of the data (Figs. 8, 10). Over the Punjab, the heavy rainfall state (Fig. 7a) is associated with a pronounced monsoon trough that draws moisture primarily from the Arabian Sea (Fig. 19). Our results provide additional confirmation of this source region. Over the upper Yangtze, the heaviest rainfall state (Fig. 9a) is associated with an extension to the north and east of the monsoon current emanating over northern South Asia, rather than the Bay of Bengal that dominates in the climatological moisture flux picture (Fig. 20). Other wet states over the Yangtze highlight the role of transient Rossby wave patterns that may be excited downstream of the Tibetan plateau.

The estimated state sequences (Figs. 8, 10) express synoptic-scale variability across time scales from subseasonal to interdecadal, as well as the seasonality of rainfall within the JJAS season. In simulation mode, however, the HMM contains no exogenous variables, and its seasonally dependent Markov transition matrix can only generate variability on the daily time scale. The cases presented here encode interannual variability solely through the regional-scale SAI variable, while subseasonal variability is generated by the HMM. This distinction is used for simplicity here and could be relaxed in future work, both by allowing the exogenous variables to encode subseasonal variability, such as associated with intraseasonal oscillations that contain predictability, as well as allowing a nonhomogeneous HMM component.

One of the motivations for constructing a GLM–HMM in which the exogenous variables act only via the GLM component (and not via modulating state transition as in the NHMM) was to test the validity of this alternative approach. A truly surprising result of this work is just how similar the interannual performances of these two approaches turned out to be for the rainfall statistics considered (Fig. 17). There are physical arguments for both paradigms of interannual climate variability. In the (loosely speaking) “linear” paradigm, a separation of times scales is made between fast “weather noise” and slower climatic influences such as ENSO; observed interannual variability is then assumed to be the sum of a stationary daily stochastic process and interannual forcing (e.g., Charney and Shukla 1981; DelSole and Feng 2013). This directly motivates the GLM–HMM approach. The alternative “nonlinear” paradigm argues that there is no separation of scales and that interannual variability at local scale results from a modulation of nonlinear weather regimes that form the chaotic attractor of weather and climate variability (e.g., Ghil and Childress 1987; Palmer 1998). This physical model has motivated the use of the NHMM in climate studies of daily rainfall sequences (Robertson et al. 2004). There is indeed evidence that drought years over India are associated with more monsoon breaks and that these are well captured by an NHMM (Greene et al. 2008). However, for climate change downscaling, the fixed nature of the NHMM states was found to be a limitation (Greene et al. 2008). Thus, a combined GLM–NHMM is advocated and this is the subject of our current work.

There are a variety of other extensions of the model that could be pursued. For example, the exogenous variables in the GLM–HMM could be station specific in order to encode additional spatial information, unlike in the NHMM approach; as in the case of the NHMM, they can also have higher frequency in time. The poorer results for the upper Yangtze basin may be due to the larger size of the region or to the more complex nature of the circulation patterns there (Fig. 9a). The ability of the GLM–HMM to encode GLM predictors at the station scale provides the potential to include smaller spatial (or temporal) scale information from a high-resolution GCM if the latter are shown to be reliable. Thus, there is considerable flexibility in how the GLM–HMM approach can be used for GCM downscaling. The model could also be extended to include an NHMM in place of the HMM, thus allowing exogenous variables to modulate the state transitions as well. Finally, while gamma distributions are shown to yield quite reasonable 50-yr return levels for the Punjab case, extremes in the upper Yangtze are less well captured, for which heavy-tailed distributions could be included in the mixture model.

In conclusion, the Bayesian GLM–HMM developed and tested here shows promise as a downscaling methodology for station rainfall and has several potential advantages compared to existing non-Bayesian NHMMs, through explicit estimates of parameter uncertainty and the model’s ability to modulate simulated rainfall distributions as a direct function of exogenous predictors. This latter feature may be particularly useful in climate change downscaling studies, where state stationarity assumptions pose an impediment for NHMMs.

We are grateful to two anonymous reviewers whose insightful and constructive comments contributed to the revised version of the manuscript. This work was supported by the U.S. Department of Energy Grant DE-SC0006616, as part of the Earth System Models (EaSM) multiagency initiative.

# APPENDIX

## Bayesian Sampling of the GLM–HMM

In this appendix, we provide mathematical details for constructing the GLM–HMM in a Bayesian framework. The simplest form of MCMC is Gibbs sampling because they are simple to sample from and do not require tuning. But Gibbs sampling requires all posterior full conditionals to be well-known distributions. To do this type of sampling for the GLM–HMM, we must introduce two sets of latent parameters: one set for the mixing weights of the emission distributions

*γ*that is used to determine the break points between the normally distributed

*t*. The transition probabilities

*q*can best be described as a matrix where

*k*th row of the transition matrix:There is one restriction: all rows sum to one,

### Algorithm considerations

#### 1) MCMC algorithm details

The algorithm implemented here is a blocked Gibbs sampler. The MCMC chain can be started in multiple locations to ensure that convergence is not to a local mode. Two hundred iterations are treated as burn in and not used to calculate posterior means; we use 2000 samples for all of the plots. However, most of the parameters’ posterior full conditional distribution become stationary after just a few iterations.

#### 2) Missing data

#### 3) Replicates

## REFERENCES

Albert, J. H., , and Chib S. , 1993: Bayesian analysis of binary and polychotomous response data.

,*J. Amer. Stat. Assoc.***88**, 669–679, doi:10.1080/01621459.1993.10476321.Ambrosino, C., , Chandler R. E. , , and Todd M. C. , 2014: Rainfall-derived growing season characteristics for agricultural impact assessments in South Africa.

,*Theor. Appl. Climatol.***115**, 411–426, doi:10.1007/s00704-013-0896-y.Bellone, E., , Hughes J. P. , , and Guttorp P. , 2000: A hidden Markov model for downscaling synoptic atmospheric patterns to precipitation amounts.

,*Climate Res.***15**, 1–12, doi:10.3354/cr015001.Charles, S. P., , Bates B. C. , , and Hughes J. P. , 1999: A spatiotemporal model for downscaling precipitation occurrence and amounts.

,*J. Geophys. Res.***104**, 31 657–31 669, doi:10.1029/1999JD900119.Charney, J. G., , and Shukla J. , 1981: Monsoon dynamics.

*Predictability of Monsoons*, J. Lighthill and R. Pearce, Eds., Cambridge University Press, 99–109.Cox, D. R., 1971:

*The Analysis of Binary Data.*Methuen, 142 pp.DelSole, T., , and Feng X. , 2013: The “Shukla–Gutzler” method for estimating potential seasonal predictability.

,*Mon. Wea. Rev.***141**, 822–831, doi:10.1175/MWR-D-12-00007.1.Forney, G. D., Jr., 1973: The Viterbi algorithm.

,*Proc. IEEE***61**, 268–278, doi:10.1109/PROC.1973.9030.Furrer, E., , and Katz R. , 2007: Generalized linear modeling approach to stochastic weather generators.

,*Climate Res.***34**, 129–144, doi:10.3354/cr034129.Ghil, M., , and Childress S. , 1987:

*Topics in Geophysical Fluid Dynamics: Atmospheric Dynamics, Dynamo Theory and Climate Dynamics.*Springer-Verlag, 512 pp.Greene, A. M., , Robertson A. W. , , and Kirshner S. , 2008: Analysis of Indian monsoon daily rainfall on subseasonal to multidecadal time-scales using a hidden Markov model.

,*Quart. J. Roy. Meteor. Soc.***134**, 875–887, doi:10.1002/qj.254.Greene, A. M., , Robertson A. W. , , Smyth P. , , and Triglia S. , 2011: Downscaling projections of the Indian monsoon rainfall using a non-homogeneous hidden Markov model.

*Quart. J. Roy. Meteor. Soc.*,**137B**, 347–359, doi:10.1002/qj.788.Hansen, J. W., , Challinor A. , , Ines A. , , Wheeler T. , , and Moron V. , 2006: Translating climate forecasts into agricultural terms: Advances and challenges.

,*Climate Res.***33**, 27–41, doi:10.3354/cr033027.Hay, L. E., , McCabe G. J. Jr., , Wolock D. M. , , and Ayers M. A. , 1991: Simulation of precipitation by weather type analysis.

,*Water Resour. Res.***27**, 493–501, doi:10.1029/90WR02650.Hughes, J. P., , and Guttorp P. , 1994a: A class of stochastic models for relating synoptic atmospheric patterns to regional hydrologic phenomena.

,*Water Resour. Res.***30**, 1535–1546, doi:10.1029/93WR02983.Hughes, J. P., , and Guttorp P. , 1994b: Incorporating spatial dependence and atmospheric data in a model of precipitation.

,*J. Appl. Meteor.***33**, 1503–1515, doi:10.1175/1520-0450(1994)033<1503:ISDAAD>2.0.CO;2.Hughes, J. P., , Guttorp P. , , and Charles S. P. , 1999: A non-homogeneous hidden Markov model for precipitation occurrence.

,*J. Roy. Stat. Soc.***48C**, 15–30, doi:10.1111/1467-9876.00136.Immerzeel, W. W., , van Beek L. P. H. , , and Bierkens M. F. P. , 2010: Climate change will affect the Asian water towers.

,*Science***328**, 1382–1385, doi:10.1126/science.1183188.Johnson, G., , Hanson C. , , Hardegree S. , , and Ballard E. , 1996: Stochastic weather simulation: Overview and analysis of two commonly used models.

,*J. Appl. Meteor.***35**, 1878–1896, doi:10.1175/1520-0450(1996)035<1878:SWSOAA>2.0.CO;2.Joshi, V., , and Rajeevan M. , 2006: Trend in precipitation extremes over India. NCC Research Rep. 3/2006, 25 pp.

Kalnay, E., and Coauthors, 1996: The NCEP/NCAR 40-Year Reanalysis Project.

,*Bull. Amer. Meteor. Soc.***77**, 437–471, doi:10.1175/1520-0477(1996)077<0437:TNYRP>2.0.CO;2.Katz, R., , and Glantz M. , 1986: Anatomy of a rainfall index.

,*Mon. Wea. Rev.***114**, 764–771, doi:10.1175/1520-0493(1986)114<0764:AOARI>2.0.CO;2.Kenabatho, P. K., , McIntyre N. R. , , Chandler R. E. , , and Wheater H. S. , 2012: Stochastic simulation of rainfall in the semi-arid Limpopo basin, Botswana.

,*Int. J. Climatol.***32**, 1113–1127, doi:10.1002/joc.2323.Kirshner, S., , Smyth P. , , and Robertson A. W. , 2004: Conditional Chow–Liu tree structures for modeling discrete-valued vector time series.

*Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence*, AUAI Press, 317–324.McCullagh, P., , and Nelder J. , 1989:

*Generalized Linear Models*. Chapman and Hall, 532 pp.Moron, V., , Robertson A. W. , , Ward M. N. , , and Camberlin P. , 2007: Spatial coherence of tropical rainfall at the regional scale.

,*J. Climate***20**, 5244–5263, doi:10.1175/2007JCLI1623.1.CPC/NCEP, 1987: CPC global summary of day/month observations, 1979-continuing. Research Data Archive at the National Center for Atmospheric Research, Computational and Information Systems Laboratory, accessed 20 May 2014. [Available online at http://rda.ucar.edu/datasets/ds512.0.]

Paap, R., , and Frances P. H. , 2000: A dynamic multinomial probit model for brand choice with different long-run and short-run effects of marketing-mix variables.

,*J. Appl. Econ.***15**, 717–744, doi:10.1002/jae.580.Palmer, T. N., 1998: Nonlinear dynamics and climate change: Rossby’s legacy.

,*Bull. Amer. Meteor. Soc.***79**, 1411–1423, doi:10.1175/1520-0477(1998)079<1411:NDACCR>2.0.CO;2.Robertson, A. W., , Kirshner S. , , and Smyth P. , 2004: Downscaling of daily rainfall occurrence over northeast Brazil using a hidden Markov model.

,*J. Climate***17**, 4407–4424, doi:10.1175/JCLI-3216.1.Robertson, A. W., , Kirshner S. , , Smyth P. , , Charles S. P. , , and Bates B. C. , 2006: Subseasonal-to-interdecadal variability of the Australian monsoon over North Queensland.

,*Quart. J. Roy. Meteor. Soc.***132**, 519–542, doi:10.1256/qj.05.75.Robertson, A. W., , Moron V. , , and Swarinoto Y. , 2009: Seasonal predictability of daily rainfall statistics over Indramayu district, Indonesia.

,*Int. J. Climatol.***29**, 1449–1462, doi:10.1002/joc.1816.Schwarz, G., 1978: Estimating the dimension of a model.

,*Ann. Stat.***6**, 461–464, doi:10.1214/aos/1176344136.Scott, S., 2002: Bayesian methods for hidden Markov models: Recursive computing in the 21st century.

,*J. Amer. Stat. Assoc.***97**, 337–351, doi:10.1198/016214502753479464.Shirley, K. E., , Small D. S. , , Lynch K. G. , , Maisto S. A. , , and Oslin D. W. , 2010: Hidden Markov models for alcoholism treatment trial data.

,*Ann. Appl. Stat.***4**, 366–395, doi:10.1214/09-AOAS282.Spiegelhalter, D. J., , Best N. G. , , Carlin B. P. , , and van der Linde A. , 2002: Bayesian measures of model complexity and fit.

,*J. Roy. Stat. Soc.***64B**, 583–639, doi:10.1111/1467-9868.00353.Timbal, B., , Hope P. , , and Charles S. , 2008: Evaluating the consistency between statistically downscaled and global dynamical model climate change projections.

,*J. Climate***21**, 6052–6059, doi:10.1175/2008JCLI2379.1.Vrac, M., , and Naveau P. , 2007: Stochastic downscaling of precipitation: From dry events to heavy rainfalls.

,*Water Resour. Res.***43**, W07402, doi:10.1029/2006WR005308; Corrigendum,**44**, W05702, doi:10.1029/2008WR007083.Wilks, D., 1999a: Interannual variability and extreme-value characteristics of several stochastic daily precipitation models.

,*Agric. For. Meteor.***93**, 153–170, doi:10.1016/S0168-1923(98)00125-7.Wilks, D., 1999b: Multisite downscaling of daily precipitation with a stochastic weather generator.

,*Climate Res.***11**, 125–136, doi:10.3354/cr011125.