1. Introduction

The widespread drought conditions and fires in the western United States, frequent flooding in the central and eastern United States, and severe drought in the southeastern United States in recent years have stimulated considerable interest in looking for causal mechanisms for intra-annual, interannual, and longer-term precipitation variability. The two most widely used indices for such studies are the Palmer drought severity index (PDSI) developed by Palmer (1965) and the standardized precipitation index (SPI) developed by McKee et al. (1993). The PDSI and SPI are both computed by complicated algorithms that obscure the inherent filtering properties of the indices. An objective of this study is to develop simplified interpretations of both of these indices. We also develop a simplified interpretation of the model calibrated drought index (MCDI) that is computed by a recursion relation that has the same form as that used to compute the PDSI, but with geographically varying weighting factors (Williams et al. 2017).

The PDSI incorporates the effects of precipitation, evapotranspiration, and runoff on moisture variability. The recursive procedure of the PDSI algorithm is intended to capture an intrinsic time scale of long-term drought variability for any specific region of interest. Interpretation of the relationships between the PDSI time series in different regions is complicated by the fact that the intrinsic time scale that is determined by the algorithm can vary substantially from region to region, even for regions that are close geographically (Heim 2002; Karl 1983; Guttman 1992, 1998; Wells et al. 2004; Wu and Kinter 2009; Williams et al. 2017). A significant limitation of the PDSI is that users have no ability to control the time scale of moisture variability that is captured by the PDSI. Another limitation of the PDSI is that it is only available for the 48 contiguous states of the United States.

The SPI addresses one of the limitations of the PDSI by allowing users to compute a precipitation index for any specified time scale. The SPI is based only on the total accumulation of past precipitation over a specified time span and its relationship to historical variability for the same time period. The index for a given time scale is referred to as SPIxx, where xx is a two-digit number that corresponds to a specified accumulation period in months.

Unlike the PDSI, the SPI does not take into consideration the effects of evapotranspiration and runoff, both of which can be important contributors to drought variability, particularly during extreme events. This has been addressed by development of a standardized precipitation evapotranspiration index (SPEI) that parameterizes the effects of evapotranspiration based on average air temperature and latitude (Vicente-Serrano et al. 2010; Beguería et al. 2014). From consideration of the cases of xx = 3, 6, and 10 months for the nine representative regions of the United States that are introduced in section 2, we determined that the correlations between SPEIxx and SPIxx time series are typically higher than 0.95. The parameterized evapotranspiration evidently contributes little new information to the SPEI beyond what is already accounted for implicitly by whatever precipitation variability is correlated with air temperature in the precipitation-only effects that are considered in the SPIxx time series. There thus appears to be no practical difference between the SPEI and SPI as time series indices of short-term climate variability. The SPEI is therefore not considered further in this study.

The degree of consistency between PDSI and SPIxx time series¹ can be assessed from their cross correlations.² Because of the wide range of the intrinsic time scale of variability in the PDSI from region to region (Wu and Kinter 2009; see also section 3b), no single choice of xx gives the highest correlations of SPIxx with the PDSI for every region. We found empirically that the span that gives the SPIxx time series that is most highly correlated with the PDSI is approximately xx = 1.25τ, where τ is the regionally specific inherent exponential time scale of the PDSI determined as described in section 3b and appendix B. The correlations between the PDSI and the optimally chosen SPIxx are typically about 0.8. For the particular case of xx = 10 months, the correlations between the PDSI and SPI10 over the nine regions considered in this study range from a low of 0.65 in Oregon to a high of 0.87 in Iowa with an overall average of 0.76. In consideration of the fact that the PDSI and SPI are the most commonly used indices for studies of precipitation variability, it is rather surprising that they are not more highly correlated, even when the span xx is optimally chosen to maximize the correlation between the PDSI and SPI.

One reason the PDSI and SPI are not more highly correlated is apparent from the scatterplot comparisons in Fig. 1a for three regions in the United States. For illustration purposes, we chose a time scale of xx = 10 months for the SPIxx time series used for these scatterplots. Scatterplots for other time scales and other regions exhibit the same salient features that are evident in Fig. 1a. In all regions, the scatterplots fall approximately along straight lines, but with distinct positive biases of positive values of the PDSI and negative biases of negative values of the PDSI. A consequence of these sign-dependent biases is that the PDSI is not a single-valued function of the SPI. This can introduce ambiguities in the interpretation of the PDSI, a point that is discussed in detail in section 3a.

View Full Size

Scatterplots of each of four drought indices vs the 10-month average standardized precipitation index (SPI10) for Oregon, Iowa, and Georgia, which are representative of the diversity of precipitation variability in the nine regions considered in sections 2–4. The (a) PDSI, (b) PMDI, (c) PHDI, and (d) MCDI computed as averages of the 1/8° gridded dataset over rectangular approximations of each of the three states. The correlation between each pair of drought indices and its estimated 95% significance level are labeled in the upper-left corner of each panel. The slopes (gains) of the straight-line regressions are labeled in the lower-right corners.

Citation: Journal of Hydrometeorology 21, 9; 10.1175/JHM-D-19-0230.1

• Download Figure
• Download figure as PowerPoint slide

There are two variants of the PDSI: the Palmer hydrological drought index (PHDI; Palmer 1965; Karl 1983; Guttman 1991) and the Palmer modified drought index (PMDI; Heddinghaus and Sabol 1991; Rhee and Carbone 2007). As shown in Fig. 1b, the sign-dependent biases in the PDSI (Fig. 1a) are much smaller or nonexistent in the PMDI but only slightly smaller in the PHDI (Fig. 1c). There is a curious paucity of PHDI values in the range ±0.5 that is unique to the PHDI. While the sign-dependent biases differ for each of the three Palmer indices, their cross correlations are typically between 0.90 and 0.95. As time series indices of short-term climate variability, there thus appears to be little practical difference between the three Palmer indices. Except in Figs. 1b and 1c and in Fig. 9 below and the related discussion in sections 3a and 3b, we consider only the PDSI in this study.

The algorithm for the MCDI (Williams et al. 2017) is a simplified version of the algorithm for the PDSI and is based on estimates of precipitation, evaporation and the fraction of available moisture from a data assimilation model from 1979 to the present and from various climate datasets prior to 1979. Monthly averages of the MCDI are available on a 1/8° grid over the 48 contiguous states of the United States. As shown in Fig. 1d, another desirable feature of the MCDI is that does not exhibit the sign-dependent biases that are present in the PDSI and PHDI.

One of the objectives of this study is to understand the opposing biases of positive and negative values of the Palmer indices (especially the PDSI) and the paucity of values near zero in the PHDI. More generally, the goal of the detailed examinations of the PDSI, MCDI, and SPI time series in this study is to quantify the filtering properties of each of these moisture indices from determination of their mathematical relationships to past, present and future precipitation anomalies.

The detailed analyses of the SPI in section 2 and the PDSI and MCDI in section 3 motivate the development of a hybrid precipitation index in section 4. MATLAB and Python codes for computing the hybrid index can be obtained from the GitHub repository as described in appendix A. With appropriately matched parameters, the average of the correlations between the hybrid index and the PDSI and MCDI is nearly 0.9. The hybrid index avoids the opposing biases of positive and negative values that exist in the PDSI and PHDI. Like the SPI, the hybrid index allows users to choose the time scale of precipitation variability that is of interest, with the added advantage of avoiding the interpretational complications of the SPI and PDSI that are identified in sections 2 and 3. It is shown in section 5 that the hybrid index is the discretized solution to a differential equation that represents forcing by precipitation (and optionally, nonprecipitation effects) for a system with damped persistence over a specified time scale. An example application demonstrating the usefulness of the hybrid precipitation index is presented in a companion study of precipitation variability along the west coast of North America (Chelton and Risien 2020, hereafter Part II).

2. The standardized precipitation index

Detailed descriptions of the SPI algorithm have been given by McKee et al. (1993), Edwards and McKee (1997), López-Moreno and Vicente-Serrano (2008), and others. Calculation of an SPIxx time series for a particular region is based on precipitation statistics derived from a long-term historical data record for the region of interest. For each of the 12 individual calendar months, the historical observations of the accumulated precipitation over a user-specified time period of xx months ending with that calendar month are fitted to a two-parameter gamma distribution (McKee et al. 1993; Edwards and McKee 1997) or a Pearson Type III distribution (Guttman 1998). The fitted probability distribution for each of the 12 individual calendar months is then transformed to a Gaussian distribution with zero mean and unit standard deviation. A value of zero corresponds to the median of the accumulated precipitation for that calendar month. The SPIxx value for a particular month t_i is computed by comparing the transformed xx-month accumulated precipitation with the transformed distribution of accumulated precipitation for the corresponding calendar month. Because of the normalization to achieve a standard deviation of 1 for each of the 12 individual calendar months, the SPIxx value at time t_i corresponds to the number of standard deviations by which the transformed accumulated precipitation over the xx months ending with month t_i deviates from the historical median for that xx-month period of the calendar year.

The SPI procedure can be clarified from consideration of a specific example. Suppose the interest is in deriving an SPIxx time series for an accumulation time of xx = 6 months. The SPI06 value for June of a particular year is computed by comparing the transformed 6-month accumulation of precipitation during January–June of that year with the distribution of the transformed accumulated precipitation for all of the 6-month periods January–June in the historical data record. The SPI06 value for the next month (July) of the particular year is then computed by comparing the transformed 6-month accumulation of precipitation during February–July of that year with the distribution of the transformed accumulated precipitation for all of the 6-month periods February–July in the historical data record. The complete SPI06 time series is constructed by following this procedure for the 6-month accumulation of precipitation for each month in the data record.

An important point to bear in mind is that the SPIxx time series is purposely normalized to achieve a standard deviation of one in the transformed distribution of accumulated precipitation for each of the 12 individual calendar months. Information about absolute departures of accumulated precipitation for each month is lost because of this transformation and normalization on a calendar month basis in the SPI algorithm. The SPIxx value for any particular time t_i is thus defined to be a measure of how anomalous the accumulation of precipitation (or more precisely, the transformed precipitation) is during the xx-month period ending with month t_i compared with the normal range of variability during that xx-month period of the calendar year. As discussed below, this complicates the interpretation of an SPIxx time series as a climate index of the variability of xx-month accumulated precipitation.

To develop a simplified interpretation of SPIxx time series, we consider precipitation variability in nine different regions in the continental United States that encapsulate the diversity of precipitation characteristics found in the contiguous 48 states. Three of these regions are along the west coast: Washington (WA), Oregon (OR), and Northern California (N.CA). Three are in midcontinent: Iowa (IA), Missouri (MO), and Arkansas (AR). And three are along the East Coast: Pennsylvania (PA), North Carolina (NC), and Georgia (GA). The seasonal cycles of the monthly averages and medians of precipitation and the standard deviations of the precipitation anomalies (defined to be the deviations of the monthly averages from the seasonal cycle) by calendar month for these nine regions are shown in Fig. 2.

View Full Size

The seasonal cycles of the average and median (thick and thin solid lines, respectively) and the standard deviation (dashed lines) of precipitation anomalies by calendar month during the period January 1948–May 2017 for the nine regions considered in this study: Washington (WA), Oregon (OR), Northern California (N.CA), Iowa (IA), Missouri (MO), Arkansas (AR), Pennsylvania (PA), North Carolina (NC), and Georgia (GA). The N.CA region was defined as summarized in appendix A. The other eight regions are all statewide averages.

Citation: Journal of Hydrometeorology 21, 9; 10.1175/JHM-D-19-0230.1

• Download Figure
• Download figure as PowerPoint slide

A characteristic of the seasonal cycles in Fig. 2 that will become important in our interpretation of the SPI is that the standard deviations of the monthly anomalies have large seasonal variability in phase with the seasonal cycles of the monthly averages and medians in WA, OR, N.CA, IA, and AR. In the three west coast regions, the averages, medians and standard deviations are highest in the winter and decrease to near zero in the summer. In contrast, the phases of the seasonal cycles in IA are shifted so that the highest values occur in late spring and early summer and decrease to near zero in the winter. In AR, the seasonal cycles are semiannual with maxima in the spring and fall/winter. The standard deviations of the monthly anomalies in the other four regions (MO, PA, NC, and GA) are relatively constant year-round, despite distinct seasonal variations of the monthly averages and medians of the precipitation.

Because of the complexity of the fitting and transformation of the probability distribution in the SPI algorithm, most users compute SPIxx time series using computer software that can be downloaded from the University of Nebraska–Lincoln website.³ An examination of the mathematical relationship between the SPIxx time series and monthly precipitation anomalies (PCP) suggests, however, that SPIxx time series can essentially be computed in a much less complicated manner. This is evident from the thick colored lines in Fig. 3, which correspond to the lagged correlations between PCP and the SPIxx time series for xx = 3, 6, and 10 months for each of the nine regions considered in this study. In consideration of the complicated details of the SPI procedure summarized above, Fig. 3 reveals a perhaps surprisingly simple relationship between SPIxx time series and the past and present precipitation. For each of the nine regions and each choice of xx, the SPIxx time series is approximately equally correlated with PCP in the present month and in the preceding xx − 1 months.

View Full Size

Lagged correlations of the PCP monthly precipitation anomalies at time t + lag with the SPIxx time series at time t and the one-sided, moving average PCPxx time series at time t (thick and thin lines, respectively, which are indistinguishable for MO, AR, PA, NC, and GA) as functions of the time lag for each of the nine regions considered in this study. Negative lags correspond to PCP leading the smoothed SPIxx and PCPxx time series. The blue, green, and red lines correspond to indices SPIxx and PCPxx with averages over xx = 3, 6, and 10 months, respectively. Estimates of the 95% significance levels of the correlations range from a low of 0.065 to a high of 0.082 with an overall average of 0.072 for the nine regions, three choices of xx, and two sets of drought indices (SPIxx and PCPxx).

Citation: Journal of Hydrometeorology 21, 9; 10.1175/JHM-D-19-0230.1

• Download Figure
• Download figure as PowerPoint slide

The structures of the lagged correlations between PCP and SPIxx in Fig. 3 suggest that each SPIxx time series is essentially equivalent to a moving average. This can be investigated from consideration of the one-sided, M-point uniformly weighted average defined by

where x₁(t_i) is the PCP precipitation anomaly during month t_i = iΔt for a sample interval of Δt = 1 month and the weights w_m for uniform weighting of past and present precipitation are

The subscript 1 on the precipitation anomalies x₁(t_i) in (1) is for consistency with the notation used later in section 3b. With the normalization factor M in (2), the M weights in (1) sum to 1 and a constant value of x₁(t_i − mΔt) for m = 0, 1, …, M − 1 is unaltered in the uniformly weighted average z_M(t_i). This normalization assures that the time series z_M(t) is an unbiased smoothed version of x₁(t) with the amount of smoothing determined by the span (M − 1)Δt of the moving averages (1).

The lagged correlations between the PCP values x₁(t) and the one-sided moving averages z_M(t) are shown for M = 3, 6, and 10 by the thin colored lines in Fig. 3. We will refer to these three time series as PCPxx, where xx = M. For five of the nine regions (MO, AR, PA, NC, and GA), the thin lines in Fig. 3 are indistinguishable from the thick lines that are the previously discussed lagged correlations between the SPIxx and PCP monthly precipitation anomalies for the values of xx = 3, 6, and 10 months. The lagged correlations of PCPxx with PCP for the other four regions (WA, OR, N.CA, and IA) are all very similar to those between the corresponding SPIxx and PCP, but with somewhat higher correlations for PCPxx.

The close similarities of the lagged correlations of PCP with the one-sided moving averages PCPxx and the SPIxx time series in Fig. 3 suggest that the PCPxx and SPIxx time series are very similar to each other, aside from different normalizations. This is confirmed in Fig. 4 from the high cross correlations between PCPxx and the corresponding SPIxx time series for all three choices of xx considered here. The correlations for the five regions MO, AR, PA, NC, and GA exceed 0.978 in all cases. The correlations are somewhat lower for the other four regions, which are the same four regions with discrepancies between the lagged correlations of PCP with the PCPxx and SPIxx time series in Fig. 3. Note, however, that the correlations between PCPxx and the corresponding SPIxx time series in Fig. 4 improve with increasing xx. For N.CA, for example, the correlations for xx = 3, 6, and 10 months are 0.835, 0.916, and 0.984, respectively. The dependence of the correlations between PCPxx and SPIxx on xx is examined in more detail later.

View Full Size

The maximum correlations between the SPIxx time series and the one-sided, moving average PCPxx time series (which all occur at zero lag) for averages over xx = 3, 6, and 10 months (blue, green, and red lines, respectively) based on year-round statistics for the nine regions considered in this study. The thin dashed lines are estimates of the 95% significance levels.

Citation: Journal of Hydrometeorology 21, 9; 10.1175/JHM-D-19-0230.1

• Download Figure
• Download figure as PowerPoint slide

The correlations in Fig. 4 are based on year-round statistics. Segregating the correlations by calendar month provides insight into the reason for the lower correlations for WA, OR, N.CA, and IA. For all nine regions, the correlations are very nearly perfect for each of the 12 individual calendar months. The results are shown for N.CA, IA, and GA in the top panels of Fig. 5. The correlations based on year-round statistics shown previously in Fig. 4 are lower for N.CA and IA (and WA and OR, not shown in Fig. 5) because of the seasonal variations of the standard deviations of PCPxx by month for each of the three choices of xx considered here (see the middle row of panels of Fig. 5). These seasonal variations are expected because of the strong seasonal variations of the standard deviations of the monthly anomalies of PCP in these four regions (see Fig. 2). With increasing averaging time xx, the dynamic ranges of the seasonal variations of the standard deviations of PCPxx decrease and the extrema of the seasonal variations shift to progressively later months because of the asymmetric one-sided smoothing of previous monthly PCP anomalies (see the middle row of panels of Fig. 5).

View Full Size

(top) The maximum correlations by month between the SPIxx and PCPxx indices (which all occur at zero lag) for averages over xx = 3, 6, and 10 months (blue, green, and red lines, respectively) for three regions (N.CA, IA, and GA) that are representative of the ranges of seasonality of the statistics for the nine regions considered in this study. (middle) The standard deviations of PCPxx by month. By definition, the standard deviation of every index SPIxx is 1 for each month, shown by the black dashed line in each panel in the middle row. (bottom) The regression slopes (gain factors) by month. Estimates of the 95% significance levels of the correlations in the top row for each region and each choice of xx are essentially the same as those shown by the dashed lines in Fig. 4.

Citation: Journal of Hydrometeorology 21, 9; 10.1175/JHM-D-19-0230.1

• Download Figure
• Download figure as PowerPoint slide

In contrast to the strong seasonal variations of the standard deviation of PCPxx in WA, OR, N.CA, and IA, there are no seasonal variations of the standard deviations of SPIxx for any of the nine regions (see the black dashed lines in the middle row of panels of Fig. 5 for the cases of N.CA, IA, and GA). This is because the SPI algorithm explicitly normalizes the transformed accumulated precipitation values to have unit standard deviation for each individual calendar month as discussed previously. The slopes of the regressions of SPIxx onto PCPxx are therefore necessarily higher when the monthly PCPxx standard deviation is low (late summer/early fall for WA, OR, and N.CA and late winter/early spring for IA) and lower when the monthly PCPxx standard deviation is high (winter/spring for WA, OR, and N.CA and summer for IA). The regression slopes thus vary seasonally out of phase with the seasonal variations of the standard deviation of PCPxx (cf. the middle and bottom panels of Fig. 5 for N.CA and IA).

The case of GA is included in Fig. 5 to show an example of a region for which the correlation between SPIxx and PCPxx based on year-round statistics is high in Fig. 4. Correspondingly, there is very little seasonal variability of the standard deviations of PCPxx or the regression slopes for GA for any of the choices of xx shown in Fig. 5 (see the middle- and bottom-right panels, respectively). The same is true for MO, AR, PA, and NC (not shown in Fig. 5), where the correlations between SPIxx and PCPxx based on year-round statistics are also high in Fig. 4 and the seasonal variations of the standard deviations of monthly anomalies of PCP in Fig. 2 are small. Not surprisingly, the approximately constant standard deviations of PCPxx decrease and the regression slopes increase with increased averaging time xx in all five of these regions (see the middle-right panel of Fig. 5 for GA).

The large seasonal variations of the regression slopes are the reason for the lower correlations in Fig. 4 based on year-round statistics for WA, OR, N.CA, and IA. This is visually apparent from the scatterplots based on year-round statistics in Figs. 6a and 6b for N.CA and IA, respectively, for the averaging times of xx = 3, 6, and 10 months considered here. The “butterfly patterns” occur because of the month-to-month variation of the regression slopes in the middle row of panels of Fig. 5. These butterfly patterns become narrower with increasing xx, as expected because of the decreasing dynamic range of the seasonal variability of the regression slopes with increasing xx noted previously from the bottom panels of Fig. 5. The black dots in Figs. 6a and 6b are the scatterplots for the calendar months of February and August, which correspond approximately to the extrema of the seasonal cycles of the regression slopes for N.CA and IA for small xx. The points become butterfly patterns when all 12 of the months are considered but are tightly clustered for each individual calendar month, thus explaining the high correlations when the statistics are stratified by calendar month as shown in the top panels of Fig. 5. Analogous scatterplots are shown for GA in Fig. 6c where there is very little seasonal variability of the correlations or the regression slopes.

View Full Size

Scatterplots of the SPIxx vs PCPxx indices for (a) N.CA, (b) IA, and (c) GA based on statistics computed only from February and August values of the indices (black dots) and on statistics computed from the other 10 months (colored dots). Results are shown for averages over xx = 3, 6, and 10 months (top, middle, and bottom rows, respectively).

Citation: Journal of Hydrometeorology 21, 9; 10.1175/JHM-D-19-0230.1

• Download Figure
• Download figure as PowerPoint slide

It is noteworthy that the small negative curvatures in each of the scatterplots in Fig. 6 indicate that positive PCPxx anomalies are slightly underestimated by the SPIxx anomalies and negative PCPxx anomalies are slightly underestimated in magnitude by the SPIxx anomalies. The curvatures are likely related to the previously discussed complicated SPI procedure that involves fitting of the distributions of PCP anomalies for each month to a two-parameter gamma distribution or a Pearson Type III distribution and then transforming to a Gaussian distribution.

Another point to be noted from the scatterplots in Fig. 6 is that there is no evidence of the offsets that were apparent in the scatterplots of the PDSI versus SPIxx indices in Fig. 1a. This indicates that those offsets are attributable to the PDSI procedure rather than the SPIxx procedure. This conclusion is further supported by Fig. 14 below.

Based on the analysis above, we conclude that the SPIxx time series are very similar to the simple one-sided moving average PCPxx time series defined by (1), except with the normalization to achieve unit standard deviation of the index values for each of the 12 individual calendar months. SPIxx time series have been used extensively for many years as precipitation indices. Moreover, the World Meteorological Organization has specifically recommended the SPI as the standard for determining the existence of meteorological drought (WMO/GWP 2016; see also WMO 2012). However, because of the normalization that standardizes the SPIxx time series to have unit standard deviation for each of the 12 individual calendar months, information about the actual accumulation of precipitation from a given SPIxx anomaly is removed from the SPIxx time series. Unless the standard deviation of precipitation is about the same for each of the 12 individual calendar months (as is approximately the case for the five regions MO, AR, PA, NC, and GA), a particular SPIxx value can have a very different interpretation during different times of year and can thus be a poor measure of total precipitation anomaly over the time period of interest. Notwithstanding the widespread acceptance and usage of the SPI, we believe that the less complicated interpretation of one-sided moving average PCPxx time series makes them preferable to the SPIxx time series, at least for studies of climate variability on intra-annual time scales. (As discussed below, the SPIxx and PCPxx time series are essentially equivalent for time scales longer than about xx = 10 months.)

The difficulty interpreting the SPIxx noted above can be illustrated from the following example. Consider the SPI03 index for N.CA. An SPI03 value of 2 is indicative of an exceptionally large 3-month accumulation of precipitation compared with the normal range of variability for that 3-month time period. During winter and spring when the average, median and standard deviation of N.CA precipitation over the preceding 3-month period are large, an SPI03 value of 2 would correspond to an accumulated precipitation amount that is approximately two standard deviations above the median value. The same SPI03 value during July–September when the average, median and standard deviation of N.CA precipitation over the preceding 3-month period are small would imply that the amount of accumulated precipitation was highly unusual for that 3-month time period. Without the context of the small median value and standard deviation of precipitation during that 3-month period, a high SPI03 value could easily be misinterpreted as an indication of drought relief when, in fact, it corresponds to a very small actual amount of accumulated precipitation and would therefore do little to alleviate drought conditions. The lack of information about the absolute departures of accumulated precipitation from the median value in an SPIxx time series thus leads to an ambiguity in the interpretation of a given SPIxx value.

While a PCPxx time series derived from (1) and (2) characterizes the actual amounts of anomalous accumulated precipitation in a specific region, it is noteworthy that the resulting time series could be easily transformed into a metric very similar to the SPIxx time series if the interest is in assessing the degree to which the anomalous accumulated precipitation is unusual in a specific region. This can be achieved by simply normalizing the PCPxx value for each particular month in the data record by the historical standard deviation of accumulated precipitation for that particular calendar month.⁴ Because of the complications of probability fitting and transformation, it is more difficult to “unstandardize” an SPIxx time series to transform it to an index of the actual amounts of anomalous accumulated precipitation in each month of the time series.

As noted previously from the cases of xx = 3, 6, and 10 that were considered in Fig. 4, the correlations between SPIxx and PCPxx time series based on year-round statistics improve as the averaging time xx is increased. This is because the seasonal variations of the standard deviations of accumulated precipitation anomalies become progressively more and more smooth and converge toward a constant value with increasing accumulation time scale xx (see the middle row of panels of Fig. 5 for N.CA and IA). The dependencies of the correlations on the averaging time xx are quantified in Fig. 7 for the four regions with the lowest year-round correlations in Fig. 4 (WA, OR, N.CA, and IA). In all four cases, the correlations converge to a nearly constant value for accumulation times xx longer than about 10 months. The converged correlation values of about 0.97 and 0.96 for WA and OR, respectively, are slightly smaller than the value of 0.99 for both N.CA and IA, perhaps because of poorer fits to the probability distribution in the SPI algorithm applied to WA and OR precipitation anomalies. For most applications, however, the SPIxx and PCPxx can be considered essentially equivalent metrics for low-frequency precipitation variability for any individual region for xx greater than about 10 months. As a general-purpose time series index of climate variability of precipitation for arbitrary choice of xx, the standardization of the SPIxx values for each of the 12 individual calendar months is less desirable than the analogous PCPxx time series for the reasons discussed above. An added advantage of the PCPxx index is that it is simple to calculate since it does not require the complications of probability fitting and transformation. In view of the high correlations of the SPIxx time series with the simpler PCPxx time series, the probability fitting and transformation in the SPI algorithm are of questionable value.

View Full Size

The correlations between the SPIxx time series and the one-sided, moving average PCPxx time series as functions of the averaging time span xx ranging from 2 to 40 months based on year-round statistics for the regions WA, OR, N.CA, and IA that exhibit the strongest seasonal variability of the standard deviation of precipitation by calendar month in Fig. 2 and for which the correlations in Fig. 4 based on year-round statistics are smallest. Estimated values of the 95% significance levels of the correlations increase systematically with increasing span xx (see the cases of xx = 3, 6, and 10 shown by the dashed lines in Fig. 4), but are much smaller for all xx than any of the correlations shown in this figure.

Citation: Journal of Hydrometeorology 21, 9; 10.1175/JHM-D-19-0230.1

• Download Figure
• Download figure as PowerPoint slide

Another interpretational issue related to that noted above for an SPI time series at any individual region arises when SPI time series for two or more regions are compared. Because information about the actual accumulation of precipitation is removed from the SPI time series for each region, it is not possible to infer the relative amounts of anomalous accumulated precipitation from the SPI time series for different regions. It is only possible to assess the relative degrees to which the anomalous accumulated precipitation was unusual in the different regions compared with the historical statistics for each region separately. As described previously, the numerical value of an SPIxx time series for any particular month t_i and each specific region can be interpreted approximately as the number of standard deviations by which the accumulated precipitation anomaly for the calendar month associated with time t_i differs from the median value of the accumulated precipitation for that calendar month. This interpretation would be exact if the monthly accumulated precipitation values were Gaussian distributed for each of the 12 calendar months.

The difficulty interpreting the SPI time series for multiple regions can be illustrated from consideration of two regions during a calendar month for which the standard deviation of historical precipitation accumulation for that month is small in one region and large in the other. The same anomalous accumulated precipitation amount in both regions would have an SPIxx value with a much larger magnitude in the region with small historical standard deviation, despite the absolute precipitation amount being the same for both regions. SPIxx values for multiple regions thus yield information only about how unusual the monthly precipitation amounts are compared with historical ranges of variability in the different regions. While this may be the question of interest in some applications, the lack of information about the actual amounts of accumulated precipitation in the SPIxx time series precludes the possibility of comparing the relative amounts of precipitation surplus or deficit in different regions. An important point to note is that this limitation in the interpretation of the SPIxx time series in multiple regions applies regardless of the value of the accumulation time xx, even for the values of xx longer than about 10 months for which the SPIxx and PCPxx for any individual region are highly correlated as discussed above.

While the discussion above favors the PCPxx time series as climate indices of precipitation variability, the PCPxx and SPIxx time series both share in common an undesirable characteristic that becomes evident from their representations in the frequency domain. By definition, the PCPxx time series consist of one-sided moving averages of PCP anomalies. By inference from the analysis above, SPIxx time series essentially also consist of one-sided moving averages of PCP anomalies, albeit complicated by the normalization to achieve a standard deviation of one for each of the 12 individual calendar months. The inferior filtering properties of the moving average smoother are well known. The filter transfer function of moving averages with a span of xx has large-amplitude sidelobes at frequencies higher than the half-power filter cutoff frequency of the moving average smoother, which is f_c = 0.443/xx cycles per month (cpm). The extrema of these sidelobes are separated by nodes in the filter transfer function that occur at frequencies that are integer multiples of 1/xx cpm.

The effects of the filter sidelobes of the moving average smoother can be seen from the frequency spectra of the SPIxx and PCPxx time series for averaging times of xx = 3, 6, and 10 months that are overlaid in Fig. 8 for the nine regions considered in this study. The “scalloped” structures separated by “notches” centered on the nodes of the filter transfer function are undesirable artifacts in the frequency content of the smoothed time series. In particular, the precipitation variability at frequencies near the notches is mostly eliminated while substantial high-frequency content in the frequency bands between the notches is passed by the moving average smoother.

View Full Size

Frequency spectra of the (left) SPIxx and (right) PCPxx indices for averages over xx = 3, 6, and 10 months (top, middle, and bottom rows, respectively). The spectra for all nine of the regions considered in this study are overlaid in each panel. Before computing each spectrum, the time series was tapered with a Bartlett window to reduce spectral leakage. The raw spectral estimates were band averaged to obtain 20 degrees of freedom.

Citation: Journal of Hydrometeorology 21, 9; 10.1175/JHM-D-19-0230.1

• Download Figure
• Download figure as PowerPoint slide

A better choice of index of precipitation variability that is similar mathematically to the PCPxx index but avoids sidelobes in the filter transfer function will be introduced later in section 4. An added benefit of this modified PCPxx index is that it has an exponentially fading memory of past precipitation that is more realistic physically (see section 5) than the one-sided, uniformly weighted block averages that define the PCPxx time series and effectively also define the SPIxx time series.

3. The PDSI and the MCDI

b. A simplified interpretation of the PDSI

Notwithstanding its shortcomings as summarized in section 3a, the PDSI has been used extensively in time series analyses of drought and precipitation variability. An objective of this section is to develop an understanding of the PDSI from a detailed analysis of the mathematical properties of the PDSI time series for the same nine regions considered in section 2. Analogous to the analysis of the SPI in section 2, insight into the nature of the PDSI for each of these regions can be gained from the black lines in Fig. 9 that are the lagged correlations of the PDSI with PCP monthly precipitation anomalies. In all nine regions, the structures of the lagged correlations suggest that the PDSI is approximately a one-sided, exponentially weighted average of past and present precipitation with a decay time scale that differs from region to region.

View Full Size

Lagged correlations of each of the three Palmer drought indices (PDSI, PMDI, and PHDI, shown as black, blue, and red lines, respectively) at time t with PCP monthly precipitation anomalies at time t + lag for each of the nine regions considered in this study. Negative lags correspond to PCP leading the Palmer indices. Estimates of the 95% significance levels of these correlations range from a low of 0.065 to a high of 0.084 with an overall average of 0.074 for the nine regions and three sets of drought indices (PDSI, PMDI, and PHDI).

Citation: Journal of Hydrometeorology 21, 9; 10.1175/JHM-D-19-0230.1

• Download Figure
• Download figure as PowerPoint slide

The blue and red lines in Fig. 9 are the lagged correlations between PCP anomalies and the PMDI and PHDI, respectively. For these other Palmer indices, the correlations with future values of PCP are lower than for the PDSI. This is because the algorithms for the PMDI and PHDI (see section 3a) exclude the backtracking that results in an implicit dependence of the PDSI on future precipitation. In the case of the PMDI, the structures of the lagged correlations are otherwise very similar to those for the PDSI. The lagged correlations with past PCP generally decay somewhat more slowly for the PHDI with a slightly higher correlation occurring one or two months in the past than at the zero lag of maximum correlation in the cases of the PDSI and PMDI. These features are likely related to the combined effects of the lack of backtracking and the use of the index X₃ to define the PHDI (see section 3a), even during transitions between wet and dry spells when X₃ is set to zero. Because of the overall similarities between the lagged correlations for each of the three Palmer indices and their high cross correlations of 0.90 to 0.95 noted in section 1, and to constrain the scope of this analysis, we consider only the PDSI hereafter.

An understanding of the mathematical properties of the PDSI that give rise to the lagged correlations between the PDSI and monthly precipitation anomalies in Fig. 9 can be developed by considering a time series y(t_i) that consists of a sum of two other time series x₁(t_i) and αx₂(t_i),

$y\left(t_i\right)=x_1\left(t_i\right)+\alpha x_2\left(t_i\right).$(4)

In the present context, x₁(t_i) is intended to represent the precipitation anomaly during month t_i = iΔt, where Δt = 1 month is the sample interval, and αx₂(t_i) is intended to represent the contributions of nonprecipitation effects (evapotranspiration and runoff) to moisture conditions in month t_i. Without loss of generality, the time series x₂(t_i) can be defined to have the same variance as x₁(t_i). The multiplicative factor α then controls the ratio of the variances of the contributions of x₁(t_i) and αx₂(t_i) to y(t_i). A value of α = 0 implies that only x₁(t_i) contributes to y(t_i) and a value of α = 1 implies that x₁(t_i) and x₂(t_i) contribute equally to y(t_i).

A precipitation index defined to be a two-sided weighted average of y(t_i) over a symmetric total span of 2K + 1 months can be written as

$z_{\exp }\left(t_i\right)=w_0\hspace{0.17em}y\left(t_i\right)+{\displaystyle \sum _{k=1}^K{w}_k\left[y\left(t_i-k\text{Δ}t\right)+\beta \hspace{0.17em}y\left(t_i+k\text{Δ}t\right)\right]}.$(5)

As noted above from Fig. 9, the PDSI time series appear qualitatively to consist of one-sided, exponentially weighted averages of past and present precipitation. This suggests that the weighting coefficients w_k in (5) that are pertinent to mathematical interpretation of the PDSI should depend exponentially on the time lags ±kΔt, hence the subscript “exp” on z_exp. The parameter β in (5) allows for inclusion of future values of y(t) in the precipitation index. For the case of exponential weighting with e-folding time τ that is symmetric about lag kΔt = 0, the weighting coefficients are

$w_k={\displaystyle \frac{1}{W}}\hspace{0.17em}{e}^{-\left|k\text{Δ}t\right|/\tau }.$(6)

If the normalization factor W is chosen so that the weighting coefficients w_k sum to 1, a constant value of y(t_i + kΔt) for k in the range −K ≤ k ≤ K is unaltered in the exponentially weighted average z_exp(t_i). This assures that the time series z_exp(t) is an unbiased smoothed version of y(t) with the amount of smoothing determined by the parameters τ, β, and K.

The parameter β in (5) determines the relative contributions of future versus past values of y(t) to the smoothed time series z_exp(t) at time t = t_i. A value of β = 0 constrains the smoothed time series z_exp(t_i) to depend only on past and present values of y(t) and a value of β = 1 implies that z_exp(t_i) depends symmetrically on past, present and future values of y(t). Values of β between 0 and 1 correspond to asymmetric two-sided weighted averaging with higher weighting of past observations compared with future observations.

As specified by (5), the decay rate of the exponential weighting (6) is the same for negative and positive lags. While it would be easy to modify the weighting to have asymmetric decay rates, the analysis below (see also appendix B) concludes that consideration of a symmetric decay rate is adequate with appropriately chosen combinations of the parameters τ, α, and β.

The smoothed time series z_exp(t) defined by (5) provides a moisture index with a time scale that can be controlled by specification of the e-folding time τ of the exponentially weighted smoothing, the parameter β and the span of 2K + 1 months over which the exponentially weighted averages are computed. It is shown below (see also appendix B) that the three parameters τ, α, and β in (4)–(6) can be chosen to yield a structure of the lagged correlations between the smoothed time series z_exp(t) and monthly precipitation anomalies x₁(t) that can very accurately replicate the structures in Fig. 9 of the lagged correlations between the PDSI time series and monthly precipitation anomalies x₁(t) for all nine of the regions considered in this study. The best choices of τ, α, and β depend only weakly on the truncation K of the range of lags in (5), as long as K is large enough that the exponential weights (6) decay to small values at the maximum lag KΔt. For all of the calculations below, we used a value of K = 30, which corresponds to a maximum lag KΔt that is about 3 times longer than the longest e-folding time τ considered in our simulations. From the analysis of exponentially weighted averages with larger τ in Part II, a choice of K ≥ 3τ was found to be a good rule of thumb.

The nonprecipitation effects x₂(t) on the actual PDSI are modeled in the complicated manner summarized in section 3a that cannot be controlled by users without modifying the algorithm for computing the PDSI. To simulate how nonprecipitation effects could affect the structure of the lagged correlations between the PDSI and PCP precipitation anomalies, we represented the nonprecipitation effects x₂(t) as a random process. For illustration purposes, we considered random processes with a wide range of spectral characteristics specified for simplicity as power-law dependencies on frequency f,

$S_{x_2}\left(f\right)=\nu f^{-\lambda }$(7)

for five choices of λ ranging from 0 to 2. The factor ν in (7) was chosen individually for each region to give a simulated time series x₂(t) with variance equal to that of the precipitation anomalies x₁(t) in each region. As noted previously, the factor α in (4) controls the relative weighting of precipitation x₁(t) and the nonprecipitation effects αx₂(t).

The effects of evapotranspiration and runoff on the actual PDSI are of course not random noise. However, any contributions of the effects of evapotranspiration and runoff on the PDSI that are correlated with precipitation would be accounted for in the PDSI time series without explicitly including evapotranspiration and runoff in the calculation. The contributions that are important to the PDSI are the parts of evapotranspiration and runoff that are uncorrelated with precipitation. Simulating these as a random process with a decorrelation time scale that is controlled by the parameter λ in (7) is therefore not unreasonable, with the caveat that the resulting simulations only account for the nonprecipitation effects in a statistical sense. The interest here is in how this statistical modeling of nonprecipitation effects alters the structure of the lagged correlations of the PDSI with monthly precipitation anomalies.

A systematic analysis of the effects of the four parameters τ, α, β, and λ on the lagged correlations between z_exp(t) and monthly precipitation anomalies x₁(t) is presented in appendix B for a case study consisting of Arkansas. The spectral characteristics of actual nonprecipitation effects x₂(t) on the PDSI undoubtedly vary from region to region. From a sensitivity analysis in appendix B, however, we found that lagged correlations between z_exp(t) and x₁(t) that very closely approximate the structure of the lagged correlations between the PDSI and x₁(t) can be obtained for a wide range of choices of λ by suitably choosing the parameter α in (4) (see Fig. B3).

The results obtained from application of the procedure in appendix B to each of the nine regions considered in this study are shown in Fig. 10 for the case of simulated nonprecipitation effects x₂(t) that have a “red” power-law spectrum (7) with λ = 1. The choices for the parameters τ, α, and β for this value of λ that were found by trial and error to give the best results are labeled on each panel for the nine regions. The structure of the lagged correlations between the PDSI and monthly precipitation anomalies x₁(t) is replicated with remarkable accuracy for every region. The parameter α that determines the relative contribution of nonprecipitation effects compared with precipitation varies only modestly but tends to be inversely related to the decorrelation time scale τ of the exponential weighting function (6). For the f⁻¹ spectral dependence of the random noise used for the simulations in Fig. 10, the value of α ranges from a low of 0.13 for IA to a high of 0.31 for WA. These regions correspond, respectively, to the regions with the largest and smallest values of the decorrelation time scale τ in (6) that gives a lagged correlation structure most closely resembling that of the PDSI for each region.

View Full Size

Lagged correlations of the actual PDSI at time t (black lines, the same as in Fig. 9) and the simulated PDSI z_exp(t) (red lines) at time t with PCP monthly precipitation anomalies x₁(t + lag) at time t + lag for each of the nine regions considered in this study. Negative lags correspond to PCP leading the PDSI and simulated PDSI. For each region, the statistical effects of nonprecipitation contributions αx₂(t) to the PDSI were approximated as random noise with an f⁻¹ spectrum. The parameters τ, α, and β used to compute z_exp(t) (see text) are labeled in each panel. Estimates of the 95% significance levels of these correlations range from a low of 0.062 to a high of 0.085 with an overall average of 0.075 for the nine regions and two sets of drought indices (PDSI and simulated PDSI).

Citation: Journal of Hydrometeorology 21, 9; 10.1175/JHM-D-19-0230.1

• Download Figure
• Download figure as PowerPoint slide

The parameter β that determines the relative contributions of future versus past precipitation and nonprecipitation effects on the PDSI also varies only modestly over the nine regions, ranging from a low of 0.15 for WA and N.CA to a high of 0.25 for OR, MO, AR, and NC. The exponential smoothing is thus strongly weighted toward past precipitation in all nine regions. The structure of the lagged correlations between the PDSI and monthly precipitation anomalies at positive lags can only be reproduced in our exponentially weighted average time series by including future values of precipitation and simulated nonprecipitation effects. This dependence on future values is presumably attributable to the backtracking procedure that is unique to the PDSI algorithm as discussed previously (see also Fig. 1f of Part II). The lagged correlations at positive lags are much smaller for the PHDI and PMDI (Fig. 9), neither of which include the backtracking procedure. The dependence of the PDSI on future values of precipitation and nonprecipitation effects implied by the higher correlations at positive lags and the β parameters in Fig. 10 evidently provide explicit characterization of the statistical effects of future precipitation that are implicit in the backtracking procedure.

The lagged correlations in Fig. 10 are for the specified value of λ = 1 for the parameter that defines the spectral characteristics (7) of the statistically simulated nonprecipitation effects αx₂(t) on the PDSI. The sensitivities of the choices of the parameters τ, α, and β to the specification of λ are summarized in Fig. B3 in appendix B. The parameters τ and β depend only weakly on the specification of the spectral characteristics of the noise. The parameter α, however, increases systematically with decreasing λ. This inverse dependence of α on λ is necessary in order to control the residual variance of the simulated nonprecipitation effects on z_exp(t) after exponentially weighted smoothing with the e-folding time τ. For a given λ, the parameter α tends to depend inversely on τ, as noted above from Fig. 10.

Another notable result from Fig. 10 is that the e-folding time τ of the exponential smoothing that yields a lagged correlation structure for our simulated precipitation index z_exp(t) most closely resembling the lagged correlations between the actual PDSI and precipitation anomalies varies widely from region to region. This diverse geographical variability of the time scale of the PDSI has been noted previously by Heim (2002), Karl (1983), Guttman (1992, 1998), Wells et al. (2004), Wu and Kinter (2009), Williams et al. (2017), and others. Along the west coast, τ is 3.6 months for WA, 4.3 months for OR, and 6.1 months for N.CA. In midcontinent, τ ranges from a much longer value of 10.1 months for IA to 7.9 months for MO and 5.1 months for AR. Along the east coast, τ varies from 4.7 months for PA to 4.3 months for NC and 5.1 months for GA.

It is also evident from Fig. 10 that the decorrelation time scale of the lagged correlations between the PDSI and precipitation anomalies x₁(t) depends on more than just the specified e-folding time τ of the exponentially weighted averages. The time scale τ that gives a lagged correlation between our simulated precipitation index z_exp(t) and x₁(t) most closely resembling the lagged correlations between the PDSI and x₁(t) has the same value of τ = 5.1 months for AR and GA. It is visually apparent from Fig. 10, however, that the lagged correlations decrease more slowly with increasing negative lag for GA than for AR. Perhaps not surprisingly, the time scale of the lagged correlations thus depends on the details of the precipitation anomaly time series as well as on the specified e-folding time τ of the exponentially weighted averages. We determined that the lagged correlations decrease more slowly for GA because the precipitation anomalies x₁(t) in that region are more dominated by low-frequency variability than are the AR precipitation anomalies. In particular, the spectrum of GA precipitation anomalies was found to have an approximate f^−1/4 power-law dependence on frequency, whereas the spectrum of AR precipitation anomalies is approximately f⁰.

The procedure above was applied to PDSI time series for several other regions in the 48 contiguous states where the PDSI is available. In all cases, the structures of the lagged correlations of the PDSI with monthly precipitation anomalies are qualitatively similar to those shown in Fig. 10 for the nine regions selected for this study. In particular, the lagged correlations are maximum at zero lag and highly asymmetric with exponentially decaying dependencies on precipitation at negative lags and much weaker correlations with precipitation at positive lags. In every case, the procedure summarized in appendix B gave a two-sided exponentially weighted average time series z_exp(t) that had lagged correlations with monthly precipitation anomalies very similar to those of the PDSI. This was true even for the semiarid regions of Arizona and New Mexico, as well as the humid region of Florida.

Inclusion of random noise x₂(t) to simulate the nonprecipitation contributions of evapotranspiration and runoff to the PDSI in the above calculations of z_exp(t) obviously makes no sense for construction of an actual drought index. The purpose of these simulations was to show that doing so with suitable combinations of the parameters τ, α, β, and λ results in lagged correlations between our smoothed time series z_exp(t) and the precipitation anomalies x₁(t) that can almost exactly replicate the lagged correlations between the PDSI and the same precipitation anomalies x₁(t) for each of the nine regions considered in this study. The simulated random noise thus has the same statistical effects on the structures of the lagged correlations as does inclusion of the actual nonprecipitation contributions of evapotranspiration and runoff in the calculation of the PDSI.

Instructions for obtaining MATLAB and Python codes for computing the exponentially weighted averages defined by (4)–(6) based on specified values of τ, α, β, and K and a random noise time series x₂(t) with spectral characteristics (7) for λ = 1 are given in appendix A. The code can be applied to the monthly averaged precipitation time series for any region as it was for Fig. 10 to obtain a smoothed time series that has lagged correlations with monthly precipitation anomalies that best match the lagged correlations for the PDSI for that region. Example codes are provided for the case of AR that is considered in detail in appendix B and for the cases of IA and WA for which the PDSIs have, respectively, the longest and shortest e-folding times among the nine regions considered in Fig. 10.

c. A simplified interpretation of the MCDI

The MCDI developed by Williams et al. (2017) is based on an algorithm that has the same form as the recursion relation (3) used for the Palmer indices but with geographically varying weighting factors multiplying the two terms on the right side of the equation. These weighting factors were determined by optimizing the correlation of the left side of the equation with soil moisture estimates from the Noah land surface model described by Ek et al. (2003) and Niu et al. (2011). The MCDI algorithm does not include the complexities of the parallel calculations of three wetness/dryness severity indices in the PDSI algorithm. This likely explains why the MCDI does not exhibit the opposing biases of positive and negative values seen in the PDSI (cf. Figs. 1a and 1d). Nor does the MCDI algorithm include the backtracking procedure that introduces an implicit dependence of the PDSI on future precipitation, rather than just the dependence on past and present precipitation that is explicit in the recursion relation.

Because the MCDI and PDSI algorithms have the same basic form, it can be anticipated that these two drought indices are highly correlated. This is indeed the case; for the three regions considered in Fig. 1 (OR, IA, and GA), the correlations are 0.88, 0.87, and 0.89, respectively. These correlations would be even higher if not for the sign-dependent biases in the PDSI that do not exist in the MDSI, and for the fact that the two indices are based on different precipitation and air temperature data.

It can also be anticipated that the mathematical relationships between the MCDI and monthly precipitation anomalies are at least qualitatively similar to the relationships found in section 3b for the PDSI. A complete analysis of the MCDI in all nine of the regions considered in section 3b is beyond the scope of this study. To gain insight into the relationship between the MCDI and past, present, and future precipitation, we consider the OR, IA, and GA regions that are representative of the range of precipitation conditions in the nine regions as noted previously from Fig. 2. The lagged correlations between the MCDI and monthly precipitation anomalies are shown by the green lines in the top panels of Fig. 11. The black line in each panel corresponds to the lagged correlations for the PDSI shown previously by the black lines in the OR, IA, and GA panels of Figs. 9 and 10. While the structures of the lagged correlations are similar for the MCDI and PDSI, there are several important differences. In particular, the lagged correlations with future precipitation are smaller for the MCDI than for the PDSI in all three regions. This is to be expected since the algorithm for the MCDI does not include the backtracking procedure as noted above. Another notable difference is that the correlations with present precipitation, i.e., at lag zero, are somewhat higher for the MCDI than for the PDSI.

View Full Size

(top) Lagged correlations of the MCDI (green lines) and the PDSI (black lines, the same as in Figs. 9 and 10) at time t with PCP monthly precipitation anomalies at time t + lag for OR, IA, and GA. (bottom) Lagged correlations of the actual MCDI (green lines, the same as in the top panels) and the simulated MCDI z_exp(t) (red lines) at time t with PCP monthly precipitation anomalies x₁(t + lag) at time t + lag for the same three regions as in the top panels. The statistical effects of nonprecipitation contributions αx₂(t) to the MCDI were approximated as random noise with an f⁻¹ spectrum. The parameters τ, α, and β used to compute z_exp(t) (see text) are labeled in each of the bottom panels. Negative lags in all six panels correspond to PCP leading the drought indices. Estimates of the 95% significance levels of these correlations range from a low of 0.062 to a high of 0.085 with an overall average of 0.075 for the three regions and three sets of drought indices (PDSI, MCDI, and simulated MCDI).

Citation: Journal of Hydrometeorology 21, 9; 10.1175/JHM-D-19-0230.1

• Download Figure
• Download figure as PowerPoint slide

Perhaps most interestingly, the decay time scale of the correlations at negative lags are shorter for the MCDI than for the PDSI in two of the regions (IA and GA). This is because the coefficients in the recursion relation (3) are held fixed for all regions in the PDSI algorithm but were optimally chosen region by region for the MCDI as noted above. The two algorithms can evidently identify different “intrinsic” time scales of variability for a given region.

The procedure presented in appendix B that was applied to the PDSI in the nine regions considered in section 3b was applied to calculate simulated MCDI time series in the three regions considered in this section to determine appropriate values for the parameters τ, α, and β in the exponentially weighted averages defined by (4)–(6). As for the calculations of simulated PDSI time series for Fig. 10, the nonprecipitation effects x₂(t) were simulated as random noise with the power-law spectrum (7) with λ = 1. Based on the analysis of the PDSI in appendix B (see Figs. B2 and B3), analogous results would be obtained for other choices of λ with appropriately chosen values of τ, α, and β.

The lagged correlations between the simulated MCDI time series and PCP for the best choices of τ, α, and β are shown by the red lines in the bottom panels of Fig. 11. In all three cases, the best fits were obtained with a value of zero for the parameter β that determines the relative contributions of future versus past precipitation and nonprecipitation effects on the MCDI, thus confirming that the MCDI depends only on past and present precipitation anomalies. The lagged correlations of the one-sided, exponentially weighted average simulations of the MCDI are very similar to those for the actual MCDI. The correlations at zero lag are slightly smaller for the simulated time series, but all other attributes of the lagged correlations are remarkably well replicated from the simulated time series.

Comparisons between the values of τ labeled in each of the bottom panels of Fig. 11 and the corresponding panels of Fig. 10 quantify the differences in the exponential decay time scales of the MCDI and PDSI noted previously for IA and GA. Specifically, the decorrelation time scales of the MCDI for these two regions are, respectively, about 40% and 30% shorter than those of the PDSI time series for IA and GA. The parameter α that determines the relative contributions of nonprecipitation effects and precipitation effects on the MCDI is larger for OR and IA and smaller for GA compared with the values of α for the PDSI.

As noted from the simulations of the PDSI in section 3b, simulating the nonprecipitation effects of evapotranspiration and runoff on the MCDI as random noise clearly makes no sense for construction of an actual drought index. The close agreements of the lagged correlations for the simulated MCDI and the actual MCDI in the three regions considered here indicate that the random noise has the same statistical effects on the lagged correlations as do the actual nonprecipitation effects.

4. A hybrid precipitation index

The importance of nonprecipitation effects on the calculation of the PDSI has been questioned by Guttman (1991) and Wu and Kinter (2009). Williams et al. (2017) have similarly noted that precipitation is the primary driver of drought variability, although nonprecipitation effects can be significant in extreme events. The benefits of including the nonprecipitation effects of evapotranspiration and runoff in the calculations of the PDSI and MCDI can be assessed by comparing the drought indices for each of the regions considered in sections 3b and 3c with a precipitation index z_exp(t) that is computed with the same exponential time scale⁵ that was used for the calculations in Fig. 10 and the bottom panels of Fig. 11, but with α = β = 0. The resulting simplified approximations of the PDSI and MCDI for each region thus exclude nonprecipitation effects x₂(t) and future precipitation effects x₁(t) from the calculation of the precipitation index.

The correlations between the PDSIs and the one-sided, exponentially weighted, precipitation-only time series for the nine regions considered in section 3b are shown by the solid circles in Fig. 12a. The correlation values range from a low of 0.82 for OR to a high of 0.91 for IA with an overall average of 0.87 for the nine regions. The simplified indices based only on exponentially weighted averages of past and present precipitation thus typically account for about 75% of the variance of the PDSI. As shown by the asterisks in Fig. 12a, very similar results were obtained for the correlations between the MCDIs and the one-sided, exponentially weighted, precipitation-only time series for the three regions considered in section 3c (OR, IA, and GA) and for the N.CA region considered in Part II.

View Full Size

Correlations at zero lag for each of the regions considered in this study: (a) the correlations between the PDSI and one-sided exponentially weighted averages of precipitation with α = β = 0 (see text) and the exponential time scales τ labeled in Fig. 10 (PCPexp) and (b) the correlations at zero lag between the PDSI and monthly anomalies of average air temperature. Asterisks are shown in each panel for the correlations between PCPexp and the MCDI for the three regions considered in section 3c (OR, IA, and GA) with α = β = 0 and the exponential time scales labeled in the bottom panels of Fig. 11. A fourth asterisk is shown for the N.CA region considered in Part II, for which the exponential time scale was found to be τ = 6.3 months. Note that the air temperature data used for calculations in (b) are from a different source than the air temperature data used to derive the MCDI. The dashed line in (a) indicates estimates of the 95% significance levels of the correlations. The pair of dashed lines in (b) bracket the range within which correlations are statistically nonsignificant with 95% confidence.

Citation: Journal of Hydrometeorology 21, 9; 10.1175/JHM-D-19-0230.1

• Download Figure
• Download figure as PowerPoint slide

The weak contribution of nonprecipitation effects on the PDSI and MCDI is further underscored by their low correlations with monthly average air temperature anomalies for each of the regions (Fig. 12b). Monthly averages of air temperature anomalies are used to estimate evapotranspiration in the PDSI and MCDI algorithms. The lag zero correlations are all negative with an overall average of −0.19 for the PDSIs and similar values for the MCDIs in the three regions considered in section 3c and the N.CA region considered in Part II. It thus appears that evapotranspiration effects that are not correlated with precipitation can account for less than 5% of the variances of the PDSI and MCDI in the regions considered in this study (see also Guttman 1991; Wu and Kinter 2009).

It should be noted that some of the variance of the PDSI time series that is not explained by our one-sided, exponentially weighted, precipitation-only indices is attributable to the opposing biases of positive and negative values of the PDSI that are evident in Fig. 1a (see also Fig. 14b below) and were discussed in sections 3a and 3b. Additionally, some of the unexplained variance of the PDSI is attributable to the inclusion of future precipitation effects in the PDSI that were also discussed in sections 3a and 3b but are not included in our one-sided, exponentially weighted average precipitation indices. The relatively small remaining unexplained variance includes the nonprecipitation effects that have been excluded from our simplified precipitation index.

The one-sided, exponentially weighted average precipitation index with α = β = 0 considered above has two features in common with the SPI precipitation indices that were examined in detail in section 2: 1) it is based only on a weighted average of precipitation and 2) it allows users to specify the time scale of precipitation variability, in this case by adjusting the parameter τ that defines the exponential weighting. The smoothing explicitly includes the exponentially damped weighting that is implicit in the PDSI and MCDI. The one-sided, exponentially weighted average precipitation index can thus be viewed as a hybrid of the SPI, PDSI, and MCDI moisture indices.⁶ A one-sided, exponentially weighted average hybrid precipitation index (PCPexp) can be computed from any time series of monthly averaged precipitation anomalies using the MATLAB and Python codes referred to in appendix A by setting α and β to zero and specifying the desired values for the e-folding time scale τ and the truncation K of the range of lags in (5).

It is shown in appendix C that an essentially equivalent hybrid precipitation index can be obtained somewhat more simply than the one-sided, exponentially weighted index by replacing the exponentially damped weighting with linearly damped weighting over a span of MΔt = 2τ (following the notation of appendix C). The correlations between exponentially and linearly weighted averages exceed 0.98 in all nine of the regions considered in this study. An advantage of the linear weighting is that it alleviates the relatively minor issue of having to choose a truncation value K of the lag range in the exponentially weighted average (5). More significantly, the linear weighting reduces the data loss at the beginning of the data record. The previously suggested rule-of-thumb span of 3τ for the lag range of one-sided, exponentially weighted averages results in a loss of 3τ data points. In comparison, essentially equivalent one-sided, linearly weighted averages result in a loss of only 2τ data points.

The single-parameter exponentially weighted hybrid precipitation index (5) based only on past and present precipitation, i.e., with α = 0 in (4) and β = 0 in (5), or its linearly weighted equivalent (C1), are simple to calculate compared with the very complicated PDSI algorithm and the somewhat complicated MCDI algorithm. They have the distinct advantage that the time scale of precipitation variability that is of interest can be controlled by specifying the time scale τ of the exponentially weighted smoothing in (6) or the span MΔt of the linearly weighted smoothing in (C2). These precipitation indices are qualitatively similar to the SPIxx and PCPxx indices that were examined in detail in section 2. The decaying memory with increasing negative lag in the one-sided, exponentially weighted averages (5) and their linearly weighted average equivalent (C1) is more physical than the uniform weighting of past and present precipitation in the SPIxx and PCPxx indices (see section 5).

Like the SPI, PDSI, and MCDI, the hybrid index z_exp(t) is a low-pass filter. The filtering properties of the PDSI and MCDI are not controllable. Analogous to the ability to control the filtering properties of the SPI by the choice of the accumulation time scale xx, the filtering properties of the hybrid index are controlled by the parameter τ [and the parameters α and β, if nonzero values are used to characterize nonprecipitation effects and the dependence of z_exp(t) on future precipitation anomalies]. By design, every low-pass filtered time series retains whatever low-frequency variability or secular trend exist in the unsmoothed monthly precipitation anomalies. If that low-frequency variability is not of interest, the SPI, PDSI, MCDI, or hybrid index must be high-pass filtered and/or detrended to attenuate the unwanted low-frequency variability.

A desirable characteristic that is shared in common by the PDSI and both the exponentially and the linearly weighted hybrid indices is evident from Fig. 13. The frequency spectra of all three of these indices do not exhibit the scallops and notches in the spectra of SPIxx and PCPxx time series in Fig. 8 that arise from the sidelobes of the filter transfer function of the uniformly weighted averages. The frequency spectra of the MCDI time series (not shown here) also do not exhibit the undesirable features of the spectra of the SPIxx and PCPxx time series.

View Full Size

Frequency spectra for each of the nine regions considered in this study: (a) the PDSI indices, (b) one-sided exponentially weighted averages of precipitation with α = β = 0 and the exponential time scales τ labeled in Fig. 10 (PCPexp), and (c) one-sided linearly weighted averages of precipitation with α = β = 0 and the spans MΔt = 2τ (PCPlin). The spectra for all nine of the regions are overlaid in each panel. Before computing each spectrum, the time series was tapered with a Bartlett window to reduce spectral leakage. The raw spectral estimates were band averaged to obtain 20 degrees of freedom.

Citation: Journal of Hydrometeorology 21, 9; 10.1175/JHM-D-19-0230.1

• Download Figure
• Download figure as PowerPoint slide

The hybrid precipitation indices also provide additional evidence that the offsets in the scatterplots in Fig. 1a are attributable to biases in the PDSI time series rather than the SPI time series. There are no offsets in the scatterplots of one-sided, exponentially weighted average time series versus the SPI10 time series that are shown in Fig. 14a for OR, IA, and GA, nor for any of the other six regions considered in section 3b. In contrast, the scatter of points for each region in the scatterplots of the PDSI versus the one-sided, exponentially weighted average time series in Fig. 14b is tighter than in the scatterplots of the PDSI versus the SPI10 time series in Fig. 1a, but with the same offsets. There are no offsets in the scatterplots of the MCDI versus the one-sided, exponentially weighted average time series in Fig. 14c.

View Full Size

Scatterplots for the same three regions shown in Figs. 1 and 11 (OR, IA, and GA). (a) One-sided exponentially weighted averages of precipitation vs the SPI10 index (the PCPexp equivalent of the scatterplots for the PDSI, PMDI, PHDI, and MCDI drought indices in Fig. 1); (b) the PDSI index vs one-sided exponentially weighted averages of precipitation; and (c) the MCDI index vs one-sided exponentially weighted averages of precipitation. The exponentially weighted averages in all panels were computed based on α = β = 0 and the exponential time scales τ labeled in Fig. 10 for (a) and (b) and in the bottom panels of Fig. 11 for (c). The correlation between each pair of drought indices and its estimated 95% significance level are labeled in the upper-left corner of each panel. The slopes (gains) of the straight-line regressions are labeled in the lower-right corners.

Citation: Journal of Hydrometeorology 21, 9; 10.1175/JHM-D-19-0230.1

• Download Figure
• Download figure as PowerPoint slide

Collectively, Figs. 1a, 6, and 14 show that offsets appear only in scatterplots that compare the PDSI with other precipitation indices. The offsets are thus definitively attributable to positive biases of positive values of the PDSI and negative biases of negative values of the PDSI. We speculated in sections 3a and 3b that these sign-dependent biases are likely attributable to the truncations of the values of the wetness and dryness indices X₁ and X₂ to assure that they are always nonnegative and nonpositive, respectively, and the binary choice of X₁ or X₂ in the algorithm for the PDSI.

5. A simple physical model for a moisture index

A simple physically based model for moisture variability can be developed by considering the one-dimensional ordinary differential equation

where z(t) is intended to be a metric for moisture variability and y(t) represents the forcing. As defined by (4), the forcing y(t) includes only precipitation anomalies x₁(t) when α = 0, but it can include other effects such as evapotranspiration and runoff with appropriate definition of αx₂(t).

Setting τ = 0 in (8a) gives the “pure-forced” case of z(t) = y(t). For nonzero τ, the characteristics of the solution can be understood by rearranging (8a) into the form

In the limit as τ → ∞, this equation reduces to dz/dt = 0, which corresponds to the persistence solution z(t) = z(0), where z(0) is the initial condition. For finite τ, the solution to the unforced equation with y(t) = 0 is

which decays to zero as t → ∞ and can therefore be viewed as damped persistence with an e-folding time of τ.

In the present context, (8a) can thus be interpreted as the ordinary differential equation for a moisture index z(t) forced by y(t) with damped persistence. The general solution for z(t) for finite τ and nonzero forcing y(t) can be obtained by multiplying each term in (8a) by the integrating factor τ⁻¹e^t/τ to get

From the product rule for differentiation, the left side of this equation is the derivative of ze^t/τ. Then (10a) is equivalent to

Replacing the independent variable t with s and integrating from −∞ to time t gives

This is more easily interpreted physically by defining the transformation of variables s′ = t − s and rewriting (11a) as

This metric for moisture variability at any particular time t thus consists of an exponentially weighted average of preceding values of the forcing prior to time t. The moisture index z(t) defined in this manner preserves the mean value of the forcing since

The one-sided, exponentially weighted average z_exp(t) defined by (4)–(6) with β = 0 that was considered in section 4 is a discretized and truncated version of the continuous, semi-infinite solution (11b) to the physically based model (8) with damped persistence. The persistence time scale τ for each of the nine regions considered in this study was chosen in sections 3b and 4 to be the value that gave an exponentially weighted time series that had lagged correlations with monthly precipitation anomalies that best matched the lagged correlations of the PDSI and MCDI with the monthly precipitation anomalies in each region (see Figs. 10, B3 and the bottom panels of Fig. 11).

While the effects of precipitation, evapotranspiration, and runoff can all be included in the forcing y(t) in the physically based model (8), it was concluded in sections 3b and 4 that precipitation alone typically accounts for about 75% of the variances of the PDSI and MCDI in the regions considered in this study. Moreover, the contributions of evapotranspiration that are parameterized by air temperature anomalies and are not correlated with precipitation anomalies appear to account for less than 5% of the variances of the PDSI and MCDI (Fig. 12b). Consideration of precipitation alone by setting α = 0 in the definition (4) of the forcing y(t) and including only past and present precipitation anomalies by setting β = 0 in (5) therefore provides a simple and reasonable physically based definition of a precipitation-forced moisture index. The exponentially damped weighting of this moisture index is consistent with the relationships of the PDSI and MCDI to past and present precipitation identified in sections 3b and 3c.

6. Discussion and conclusions

The Palmer drought severity index (PDSI) and the standardized precipitation index (SPI) have both been used extensively for many years to investigate drought and precipitation variability. It is therefore surprising that the correlations between the two indices for the same region are typically only about 0.8, even when the time scale of the SPIxx index is optimally chosen to best match the PDSI. In an effort to determine the reasons for the discrepancies between the PDSI and the SPI, we conducted a detailed analysis of their mathematical relationships to monthly precipitation anomalies in nine different regions of the continental United States. Both indices can be closely approximated by weighted averages of past and present precipitation, but with different weighting. In the case of the SPI, the weighting factors are uniform over the specified span of the SPI for the time scale of precipitation variability that is of interest. In the case of the PDSI, the weighting factors decay exponentially with increasing negative lag. The algorithm for computing the PDSI implicitly includes information about future precipitation anomalies as well, but with much smaller weighting than past and present precipitation.

Another distinction between the PDSI and the SPI is that the former incorporates information about the effects of evapotranspiration, which is parameterized in terms of air temperature. The SPI has been extended to include effects of evapotranspiration in the standardized precipitation evapotranspiration index (SPEI). As noted in section 1, evapotranspiration has very little effect on the SPEI beyond what is already accounted for by precipitation since the correlations between the SPEI and SPI time series in the nine regions considered in this study are typically greater than 0.95. Evidence that evapotranspiration also has relatively little effect on the PDSI was presented in Fig. 12b (see also Guttman 1991; Wu and Kinter 2009).

The detailed analyses of the SPI and PDSI in sections 2 and 3, respectively, identified artifacts in both of these moisture indices. In the case of the SPI, the standardization of the SPI index values on a calendar month basis can complicate the interpretation of the SPI for any individual region for accumulation time scales shorter than about 10 months (see Fig. 7 and the discussion in section 2). For accumulation time scales longer than xx = 10 months, the SPIxx time series are almost perfectly correlated with uniformly weighted moving averages of precipitation, referred to here as PCPxx. Aside from different normalizations, the SPI on these longer time scales is thus essentially equivalent to a one-sided, uniformly weighted moving average, which can be calculated much more simply without the fitting and transformation of the probability distribution that are parts of the SPI algorithm.

Another limitation of the SPI that arises regardless of the value of the accumulation time scale xx is that care must be taken when interpreting the comparison of SPIxx time series for two or more regions. Because of the normalization in the SPI procedure, time series for different regions can only be used to compare the relative degrees to which the accumulated precipitation was unusual compared with the historical ranges of variability in each separate region. Since information about actual amounts of accumulated precipitation is removed from each SPIxx time series, it is not possible to use SPI time series to compare the amounts of precipitation surplus or deficit in different regions. This interpretational limitation is discussed in detail from an example in section 2 (see also section 4 and appendix B of Part II).

A peculiarity of the PDSI is that it has distinct positive biases of positive values and negative biases of negative values, resulting in offsets in scatterplot comparisons with other precipitation indices (see Figs. 1a and 14b). It is conceivable that the opposing biases of positive and negative values of the PDSI might not be a concern in applications for the purposes for which the PDSI was originally intended, namely to identify the beginning, ending and severity of droughts or wet spells. More likely, however, these sign-dependent biases are problematic even for those applications. An example presented in section 3a shows that these sign-dependent biases can result in interpretational ambiguities that raise questions about the utility of the PDSI. In any case, sign-dependent biases are clearly undesirable artifacts in time series for quantitative analyses of climate variability.

Other well-known limitations of the PDSI are the highly complicated algorithm that defines the PDSI and the fact that the algorithm determines an intrinsic time scale of drought variability with no ability for users to control the smoothing properties of the PDSI to investigate drought variability on any specific time scale of interest. One consequence of this is that the PDSI is not well suited to studies of the relationships between drought variations in different regions because of region-by-region differences between the intrinsic time scales determined by the PDSI algorithm (see e.g., the wide range of e-folding times τ of the PDSIs in Fig. 10). Moreover, the restriction of the PDSI to just a single intrinsic time scale in each region limits the utility of the PDSI for studies of drought variability since drought variations generally exhibit a wide range of time scales rather than just the single intrinsic time scale identified by the PDSI algorithm.

The MCDI developed by Williams et al. (2017) is a significant improvement over the PDSI. It is computed by the same recursion relation (3) that is used to compute the PDSI, but with geographically varying coefficients that were optimally chosen to maximize the correlation of the MCDI time series with model estimates of soil moisture. The MCDI algorithm also does not include the complexity of the parallel calculations of three wetness/dryness severity indices in the PDSI algorithm. Nor does it include the backtracking procedure of the PDSI algorithm that is intended to determine unambiguously when a dry spell or wet spell has begun or ended. As discussed in sections 3a and 3b, these complexities of the PDSI algorithm are likely responsible for the undesirable artifacts in the PDSI time series. In particular, the complicated algorithm for choosing between the three wetness/dryness severity indices is likely responsible for the sign-dependent biases in the PDSI time series, and the backtracking implicitly introduces a dependence of the PDSI on future precipitation that is physically unrealistic in an index of drought variability.

While the MCDI eliminates the above undesirable features in the PDSI, the MCDI time series for any individual region is still restricted to a single intrinsic time scale of drought variability that is identified by the MCDI algorithm. As with the PDSI, there is no ability for users to control the smoothing properties of the MCDI to investigate drought variability on any specific time scale of interest. Furthermore, the intrinsic time scale identified by the MCDI algorithm is not necessarily the same as that identified by the PDSI algorithm.

Another limitation of the PDSI and MCDI is that both of these drought indices are available only for the 48 contiguous states of the United States.

To address the above problems while retaining the essential strengths of the SPI, PDSI, and MCDI inferred from the mathematical properties of these three moisture indices, a hybrid index was introduced in section 4 that consists of one-sided, exponentially weighted averages of precipitation that are implicit in the PDSI and MCDI. Like the SPI, PDSI, and MCDI indices, the hybrid index is a meteorological drought index. In the form analyzed in section 4, this exponential weighting includes only the effects of present and past precipitation, but it can be adapted to also include nonprecipitation effects. The dependence of the hybrid index on precipitation alone is shared with the SPI, albeit with an exponentially fading memory of past precipitation that is shared in common with the PDSI and MCDI and shown in section 5 to be more physical than the uniform weighted averages of the SPI. The high correlations of the PDSI and MCDI with the hybrid precipitation index defined with appropriately matched exponential time scales indicate that the nonprecipitation effects of soil moisture and evaporation on drought variability are small in a statistical sense compared with precipitation. In extreme drought conditions, however, nonprecipitation effects can be significant.

A distinct advantage of the PCPexp hybrid precipitation index over the PDSI and MCDI is that the explicit specification of the exponential weighting allows users to control the time scale of precipitation variability that is of interest by choosing the e-folding time of the weighting. This ability to control the time scale of the hybrid index is useful because precipitation variability in any given region occurs on a wide range of time scales, not on just the single time scale identified by the PDSI or MCDI algorithms. In this sense, the hybrid index has the flexibility of the SPI, but the exponential weighting of past precipitation in the hybrid index is more desirable than the uniform weighting in the SPI as noted above. The hybrid index also does not have the interpretational complications of the SPI or PDSI. Another advantage of the hybrid index is that it is easily calculated for any region based only on a time series of precipitation anomalies. The hybrid index can thus provide a PDSI/MCDI-like drought index in regions where the PDSI and MCDI are not available, i.e., outside of the conterminous United States.

Examples of the usefulness of the hybrid index for analysis of precipitation variability are presented in a companion analysis (Part II) from an investigation of the time dependence of geographical patterns of precipitation variability along the west coast of North America and their relationships to atmospheric forcing. In addition to enabling an investigation of multiple time scales of precipitation variability for that study, the hybrid index provides PDSI/MCDI-like indices for British Columbia and Alaska where neither the PDSI nor the MCDI are available.