Over the last two decades, probabilistic weather forecasts have been developed to quantify the uncertainties inherent in modeling the climate system. The skill of these forecasts has steadily increased, but the question of whether they are usable for water resources management remains open. The interdisciplinary study described in this paper combined a modeling approach with qualitative methods to identify technical and nontechnical factors that enhance or constrain the usability of probabilistic weather forecasts for reservoir management, using a case study of drought management decision-making by a water supply company in northwestern England.
The modeling approach calibrated and applied probabilistic medium- and extended-range precipitation forecasts from the European Centre for Medium-Range Weather Forecasts (ECMWF) to a simplified water resources system to study the technical quality of the forecasts that, in principle, could have informed management decisions during a drought event in 2010. The qualitative approach comprised initial semistructured interviews with water managers and regulators and follow-up discussions using the model experiment results to elicit further insights into the potential for incorporating probabilistic forecast information into decision-making processes.
The technical analysis showed that even these postprocessed forecasts did not have skill beyond the medium range; this constrains the type of management decisions the forecasts can inform. Regulatory frameworks and attitudes to risk in the water sector also inhibit the take-up of probabilistic forecasts for drought management decisions owing to the high stakes of such decisions and considerations spanning entire water resource zones.
In recent years, the quantitatively measurable skill of weather forecasts continues to improve according to metrics that show model developers and forecasters that models have a “basic” level of skill that is constantly increasing (Bauer et al. 2015). However, the question of whether these forecasts are usable for water resources management also relies on other, nontechnical, factors.
Previous studies have explored the potential for probabilistic numerical weather forecasts and climate projections to be effectively applied in decision-making in the water resources sector (e.g., Golembesky et al. 2009; Steinschneider et al. 2012). Value-of-information approaches assert the potential for probabilistic forecasts to be integrated into tools for optimizing decisions (e.g., Palmer 2002). Other studies (using methods including ethnographic interviews and role-playing games) have demonstrated the constraints and challenges to the operational uptake of weather and climate forecasts by decision-makers in the water sector (e.g., Crochemore et al. 2015; Feldman and Ingram 2009; Rayner et al. 2005). This paper documents an interdisciplinary exercise that explored these challenges in a real-world context so that water managers and forecasters alike might better understand the technical and institutional factors influencing forecast use in the U.K. water supply sector.
In collaboration with a U.K. water company and the relevant regulatory body [the Environment Agency (EA)],1 we used both qualitative and quantitative methods to explore whether probabilistic weather forecasts can provide usable information for reservoir management decision-making. Currently, the water company uses indicators including actual (observed) reservoir levels and river flows, rainfall amounts and soil moisture deficits over the preceding period, forecast abstraction rates, and regional weather forecasts provided by the Met Office to inform its decisions about pumping and water transfers. The company uses a range of modeling tools for individual reservoirs and, regionally, in their water resource assessments to support scenario analysis. Using historical data to derive probabilistic (return period) inflows, these tools simulate reservoir and system performance over the coming year and beyond. The water company shares the results with the EA. The interpretation of the results of the risk assessments aids decision-making for the management of the water supply system (United Utilities Water 2014, p. 14). At present, the water company is not feeding numerical weather model forecasts into hydrological models.
This study combines a modeling approach with qualitative methods to explore whether probabilistic weather forecasts on the medium (1 to 10 days’ lead time) to extended range (2 to 5 weeks’ lead time) can provide information that water managers would find usable for management decision-making. To do this, we identified a past episode where the deterministic forecasts available at the time had been proved incorrect; we then explored how the forecasts provided by an ensemble prediction system could have contributed to the decision-making process and the opportunities and challenges of using this information in a context of balancing cost and risk in operational decision-making. The quantitative part of the analysis involved using probabilistic weather forecasts for the location of a subsystem of one of the company’s water resource zones to explore the added value of these forecasts to support management decisions. The qualitative research comprised two stages. In the first stage, we conducted two semistructured face-to-face group interviews: one with four water company staff who provide technical advice for water management operations (water managers) and one with two representatives from the local EA office. A member of the EA’s national climate team was present in both interviews. The interview protocol included questions about the scope, timing, and constraints of management decisions; the water resource system’s sensitivities to weather/climate; the current use of numerical weather model forecasts and other information; and the opportunities and challenges for using forecasts to inform decisions. In the second phase, we undertook iterative discussions via teleconference and e-mail with the water managers, using the results of the model experiment to elicit further insights into the real-world context of water resources management and the implications for incorporating probabilistic weather and seasonal forecast information into decision-making processes. Later sections of the paper also draw on material from additional semistructured interviews with water managers from other U.K. water companies.
We start by outlining the context of the study, including a description of the water resources system and current drought management approaches. We then apply our analysis to a situation that arose in the summer of 2010, including consideration of the weather forecasts that were available to the water managers at the time and the quality of probabilistic weather forecasts that were being produced at the time but not available to the water managers. The analysis is followed by some comments on the usability of longer-lead-time (seasonal) forecasts, and the paper concludes with a discussion of the roles of forecast attributes and of risk and regulation for understanding the challenges of operationalizing probabilistic weather forecasts for U.K. water management decision-making.
2. Context of the study
a. Water resources system
Following a series of interviews and discussions with the water company’s resource managers and with EA managers, we identified a small subsystem of one of the company’s four water resource zones2 to analyze quantitatively the potential use of probabilistic weather forecasts in drought management. As all of the variables and components of decision-making undertaken by the company cannot be accounted for in the scope of this assessment, we have simplified the chosen subsystem. The selected subsystem comprises three reservoirs/lakes (A, B, and C). Water from reservoir A is pumped directly to the Water Treatment Works (WTW) whereas water from reservoir B is pumped into the main reservoir (C) from where it is abstracted to the WTW (Fig. 1). Reservoirs A and B are about 3 and 2 times larger than reservoir C, respectively, and in both cases, the primary outflow is to a river. Depending on the drought event, the critical period3 for the entire water resource zone is 6–18 months with drawdown starting at the end of March.
The most recent severe drought affecting the water company’s region was a 2-season drought event in 1995/96.4 As noted in the water company’s 2014 drought plan, customer experiences of this drought prompted the company to improve the level of service for water supply. The new level of service calls for imposition of statutory water use restrictions and drought permits/orders5 not more than once every 20 years on average. Since the improved level of service became effective in 2000, no drought permits/orders have been implemented; one hosepipe ban was implemented for eight weeks during the summer of 2010 (United Utilities Water 2014).
Because of the length of time required to apply and obtain approval for drought permits/orders from the EA (21 to 35 days), water companies often make applications that are subsequently withdrawn or not implemented when the water resources situation improves. The water companies usually decide to apply for drought permits/orders as a precaution against continuing dry weather. For example, this occurred because of dry weather during the autumn of 2002 and into 2003 (United Utilities Water 2014) and again in 2010.
A dry spring and early summer in 2010 led to low reservoir storage in the resource zone containing the subsystem under study. The water company introduced a hosepipe ban6 on 9 July and lifted it on 19 August 2010 (United Utilities Water 2014). Moreover, on 7 July 2010, the company made a drought permit application to the EA to reduce the source A hands-off flow (required minimum flow) and to relax the rolling annual license limit. However, because of an improvement in weather conditions, the water company withdrew the permit application on 26 July 2010, and the powers were never implemented (United Utilities Water 2014).
The 2010 experience shows that applying for drought permit/orders is undertaken according to a precautionary approach in this region where the reservoirs are very “flashy”; that is, they can refill in a matter of weeks depending on the rainfall and preexisting conditions. Because of the time needed to apply for drought permit/orders, the water company needs to make applications when reservoir levels are higher than the point where the permits must be implemented. This means that the water company will apply for drought orders/powers more frequently than they will be used. This incurs a cost and highlights the potential utility of weather forecasts if they could be used to minimize the number of applications for drought permits/orders that are not used due to improvement in the weather conditions.
b. Decision triggers and actions
“Drought triggers” are decision points at which water managers evaluate the measures required to address the developing situation and decide whether or not to implement certain actions. There are four drought triggers for reservoir C. These are illustrated in Fig. 2, which shows reservoir storage as a function of time for each of the triggers (within a year); at any given time, there will be a point, that is, a particular value of the reservoir storage that, when crossed, triggers the evaluation of different options for management actions. In this diagram, the dashed line represents the point at which the company considers applying for drought permits/orders. It is expected to be approximately one week after the first actions associated with trigger 3. In realistic situations, and for each particular event, the actions considered might affect reservoirs A and B or other components of the water system; depending on the wider system state, it is also possible that the manager—in consultation with the EA—may defer the decision to apply for drought orders/permits.
Trigger 4 corresponds to “drought” status and is the point at which statutory water use restrictions are expected to be implemented. The company’s minimum level of service plans for this trigger to be reached no more than once in 20 years on average (United Utilities Water 2014, p. 72). The water company has designed drought triggers 1 to 3 to ensure that there is time to carry out the required actions before the next trigger is crossed. The frequency of each trigger being crossed is assessed by modeling using historic inflow records. For example, trigger 1 is expected to be crossed on average once in three years. Table 1 summarizes a selection of drought management actions that the water company has linked to the drought triggers for the key strategic reservoirs together with their expected frequencies.
Table 1 indicates that the time it takes for the reservoir to go from one trigger level to the next is approximately two to five weeks, depending on the trigger. For some of the management actions, the time required for implementation is about one week. The rapid response of the water resources subsystem we are studying means that medium- (1–10 days lead time) to extended-range (1 to 5 weeks) probabilistic weather forecasts provide information that could in principle be used by water managers to inform decisions about the management of the feeding reservoirs.
To explore this possibility for a particular sensitive period in the recent past, we will address two questions:
What was the technical quality of the probabilistic weather forecasts available at the time (according to some standard measure of quality, explained below)?
Can probabilistic weather forecasts (as distinct from historical information) inform the management of the system, and if so, how?
3. Analysis of the summer of 2010
In the modeling exercise, we explored the problem in hindcast mode, using archived probabilistic forecasts produced by the European Centre for Medium-Range Weather Forecasts (ECMWF), to investigate whether they could have been used for decision support in a particular sensitive period in the recent past.
At the beginning of May 2010, reservoir C level was above trigger 1 (approximately 80% storage). A period of dry weather caused it to cross trigger 1 at the end of May, then trigger 2 during the second half of June, and trigger 3 at the beginning of July, going on to briefly cross the drought permit application indicator. On 7 July, the company applied for a drought permit and on 9 July imposed a hosepipe ban. Soon after, a period of wet weather caused the storage in reservoir C to recover quickly, reaching about 70% at the beginning of August. Note that these actions were based on the 2008 Drought Plan triggers valid at the time.
When discussing this particular period, the water managers commented that when reservoir C crossed the third drought trigger (around the first week of July), an evaluation of the system together with deterministic forecasts of continuing dry weather led to the implementation of a hosepipe ban. As this measure was implemented, the weather situation changed: precipitation was higher than average during the following weeks, and the reservoirs recovered quickly.
Our discussion with the water managers suggested that the available deterministic forecast of continuing dry weather was an important factor in the decision to implement the hosepipe ban at that point. Given the information available to the water managers at the time (see below for details), the decision to implement a hosepipe ban was clearly aligned with the drought management plan’s precautionary approach—as noted in section 2a, precautionary in terms of seeking to avoid service failure by taking early action even if this means bearing the cost of initiating measures that with hindsight may be proven unnecessary. In fact, the unexpected change in the weather situation had a large reputational cost for the company, and with hindsight, costs could have been saved if the measure had not been implemented. We now explore under what circumstances a probabilistic weather forecast, which includes quantitative information about the uncertainty of the prediction, could provide usable information in this practical context.
a. Forecast and forecast skill
Water managers at the company receive monthly weather forecasts (including temperature, sunshine, and precipitation) every two weeks, with a qualitative update just for the week ahead provided in the intervening week. These forecasts include the northwest of England as one of 11 U.K. regions. The managers also receive weekly precipitation forecasts twice weekly (Monday and Thursday) with more detailed information for different areas of their water resource zone. In weekly meetings, the water managers consider the weather forecasts, together with information about the level of the reservoirs, both in the subsystem and region, and the abstraction forecast for the following week and make decisions about the operation of the water system. These decisions are implemented on the Monday of the following week.
Mimicking the decision process within the simplified subsystem, we explored what (if anything) would have been different if the water manager had access to probabilistic rainfall forecasts for the locations of reservoirs A and B to inform her decisions. We therefore analyzed the ECMWF probabilistic rainfall forecasts that were issued every Thursday in 2010 for the following 4 weeks. Archived 2010 ensemble daily total precipitation forecasts for the locations of reservoirs A and B were obtained from the ECMWF (Molteni 1996). These forecasts consist of 51 member ensembles, launched every Thursday and running up to 32 days out. We computed weekly averaged forecasts by averaging over days 4–10, 11–17, 18–24, and 25–31 of each member ensemble time series. Hereafter, we label these “week 1” to “week 4” ensemble forecasts. See the supplemental material (SM) for information on the data utilized.
We evaluated the quality7 of the ECMWF probabilistic forecast model output as measured by the continuous ranked probability score (CRPS). The CRPS is an integral measure of the quality of the forecast probability distribution, and it measures the difference between the predicted and the occurred cumulative distribution functions, being 0 for a perfect forecast (Hersbach 2000). We found that the CRPS of the ensemble precipitation forecast is larger than the CRPS of a forecast produced using only climatological (historical) information for all lead times. This implies that, in this case, the decision-maker is better off using the historical information over the direct output of the ensemble prediction system. See the SM for details on the forecast evaluation.
To obtain a forecast with increased skill, we used an approach similar to the quantile–quantile mapping commonly used in hydrological applications (Panofsky and Brier 1968; Hay et al. 2002; Wood et al. 2002, 2004), consisting of using the probabilities forecasted by the ensemble prediction system to conditionally resample from the climatological distribution (see the SM for a description of the resampling approach). By using the ECMWF forecast probabilities in this way, but taking the resampled forecast directly from the historical information, we expect the approach to improve the skill of the raw forecast by correcting the numerical forecasting model’s biases due to, for instance, mismatches in spatial scales. This approach generates a probabilistic forecast of precipitation, inflows, and reservoir levels under the assumption that the main determinant of inflows and reservoir level is the total daily precipitation. The probabilistic forecasts obtained in this way demonstrated improved skill in week 1 for precipitation; that is, the CRPS of the probabilistic forecast was smaller than the one corresponding to a climatological forecast (see SM for details on the forecast evaluation and limitations of the approach).
We use the resampled precipitation ensemble in the following analysis. We also assume that the precipitation forecasts for reservoirs A and B will drive the management decisions. This is of course only valid for our simplified system. As mentioned above, in the real situation, the water company would take into consideration information on reservoir C and the general state of the whole resource zone, together with demand forecasts,8 when operating the reservoir C.
Figures 3 and 4 illustrate the week 1 resampled precipitation forecasts (forecasts issued on a Thursday and giving the average precipitation over days 4–10, that is, the week commencing the following Sunday) for reservoirs A and B for the critical period in 2010. Probabilistic forecasts illustrated as box plots are presented together with the climatological weekly averages and actual observed values.
While the observed precipitation was below the climatological value for most of May and June, the situation changed from the last week of June. The forecast issued on 24 June for week 1 (the week commencing 27 June) has a 73% probability of drier-than-average conditions. Therefore, even though a deterministic forecast following, for example, the ensemble median (lines within the box plots in Figs. 3 and 4) would have forecast drier-than-average conditions, the probabilistic forecast indicates that there is a 27% probability of conditions being wetter than average.
How might this alter the decision-making process? When using a deterministic forecast to trigger a decision to act (e.g., apply for drought permit/orders), only two outcomes are possible: apply for the order if dry weather is forecast, and do not apply/defer the decision otherwise. On the other hand, with a probabilistic forecast, the uncertainty of the information can be taken into account by tuning the decision to act depending on the forecast probability of the weather event of interest. In other words, even though the decision at any given point is binary (act/do not act), action would be triggered only if the probability of the weather event is more than a given probability threshold.
We define the weather event as “drier-than-average conditions”; that is, the event is defined to occur if the average precipitation each week is less than the average historical value for that week. Forecast probabilities for this event were simply obtained by counting the proportion of ensemble members that forecast drier-than-average conditions as defined above. Using the resampled ensemble forecast, we computed weekly averaged probability forecasts of drier-than-average conditions for weeks 1 to 4 by averaging over days 4–10, 11–17, 18–24, and 25–31 of the resampled forecast time series, respectively.
The question is then how to determine the probability threshold at which the action is initiated. The forecast probability threshold is, in principle, chosen by optimizing a utility function that depends on the forecast quality, the cost of the action to be triggered, and the avoided losses if the action is implemented (Richardson 2003). The action is only triggered if the forecast probability is higher than the optimal threshold. The optimal probability threshold is different for different actions as it depends on the action’s cost–loss structure.
However, this assumes that the decision-maker is capable of optimal decision-making in the presence of uncertainty (represented in this case by probabilistic weather forecasts), and also that the cost–loss ratios of the management actions are known and quantifiable (in monetary terms). Quantifying that information is not trivial and usually subject to many uncertainties. For example, how to quantify in monetary terms the loss of credibility of the water company for imposing a hosepipe ban when, in the perception of the users, it is raining heavily? How to quantify in monetary terms the environmental impacts of additional abstraction or the extensive consequences for the wider economy of missing opportunities to implement management decisions that could prevent future water use restrictions or other system failures?
Since we do not have the information about costs–losses of the actions, we focus instead on estimating, for our forecasting system, how many times on average the decision-maker should expect to act in vain (false positive) or miss an event (false negative) depending on the probability threshold chosen. This can be visualized in a contingency table such as Table 2. As above, for the purposes of this exercise, we define the weather event as drier-than-average conditions. False positives are clearly linked with actions in vain since, for the chosen probability threshold, a forecast of dry weather that triggers, for instance, an application for drought powers but is followed by observed wet weather incurs costs that are (with hindsight) unnecessary. A false negative—a forecast of wet weather followed by observed dry weather—would lead to a missed event if management actions are delayed until it is too late to implement them effectively. For the water manager, and depending on the action taken, a false positive might have considerable monetary and reputational costs. However, the situation is not symmetric: a missed event has potentially much larger costs if the inaction results in the water company not being able to fulfil its statutory duty “to supply adequate quantities of wholesome water, with as little recourse as reasonably possible to drought orders or drought permits” [Water Industry Act (Stationery Office, 1991a), section 39B(2), which applies to England and Wales].9 In the interviews, water managers were clear that avoiding a failure in service was their ultimate motivation: following from this, their concern with being able to justify to the EA any recourse to drought permits highlighted the importance they placed on basing decisions on information that is recognized by the regulator. The legal criteria that underpin EA decisions to issue or decline drought permits are set out in the Water Resources Act (Stationery Office, 1991b): a permit may be issued “if the Agency is satisfied that, by reason of an exceptional shortage of rain, a serious deficiency of supplies of water in any area exists or is threatened” [Water Resources Act 1991, section 79A(1) (Stationery Office, 1991b)]. Following EA guidance,10 the permit application must provide evidence in the form of monthly rainfall figures compared with the long-term average. Water companies must also have taken demand-side measures as included in their plan or be able to justify their reasons for not doing so.
Using the week 1 precipitation forecasts for reservoirs A and B for the year 2010, we compute, for different forecast probability thresholds, how many times the drier-than-average forecast is actually followed by drier-than-average observed conditions as a proportion of the number of times that the weather verifies as dry (hits) and how many times a drier-than-average forecast is followed by wetter-than-average conditions as a proportion of the total number of drier-than-average weather forecasts (false alarms). The possible combinations of forecasts and events are illustrated in the contingency table in Table 2. The quantities computed for each of these categories will of course depend on the extreme event chosen and, for the given event, on the selection of forecast probability threshold.
The results of the calculations for two different probability thresholds are shown in Table 3. We find that for reservoir A, for instance, for a probability threshold of 70%, that is, when the forecast for drier-than-average weather is 70% or more, the expected number of hits is 81%; consequently, the proportion of missed events (which is 1 minus the number of hits) is 19%. The proportion of false alarms is 23%; therefore, the manager should expect that, on average, 23% of the times she acts, the action will be in vain if she chooses this probability threshold.
If she instead chooses 50% as the probability threshold, the expected proportion of missed events is approximately 10%, and the proportion of false alarms is 23%. It will thus make sense to choose this lower probability threshold if the aim is to minimize expected number of missed events while maintaining the same proportion of actions in vain.
In principle, for many actions, the proportion of events that can be missed, and the proportion of times the manager can act in vain, will depend on the relative costs and losses for each of these situations. In practice, if the stakes of missing an event (i.e., missing the opportunity to secure water supply) are intolerably high, the water manager will probably require a very high level of confidence in a forecast for a more extreme weather event (such as a very low probability of precipitation falling in the bottom tercile of the climatological distribution) before being willing to use a probabilistic numerical weather model forecast as a justification to override a more “precautionary” decision informed by the historically derived triggers and scenarios agreed with the EA.
If, instead of using all the probabilistic information contained in the ensemble forecasting system, the manager decides to use only the ensemble mean (a situation similar to having just the information provided by a deterministic forecast), we find that for our forecasting system and reservoir A, the expected number of missed events is approximately 57% and of false alarms is 33%. In other words, using a deterministic forecast would lead to a higher proportion of both missed events and actions in vain.
We can conclude that for this forecasting system, and assuming that the drought management decision can be based on a week 1 forecast of drier-than-average precipitation for reservoir A, the implementation of such a measure using a threshold probability of 70% can be expected to be an action in vain about 23% of the time on average. Therefore, in the context of this simplified system, had the water manager had this probabilistic forecast at the end of June 2010, she could have expected that, even with a 73% probability of drier-than-average precipitation for reservoir A, there was a 23% chance of taking drought management actions in vain. We remark that all these figures are subject to sampling uncertainty, which is expected to be large for small samples like this one. To produce robust estimates of these statistics, an analysis of a larger sample of reforecasts (for several consecutive years, for instance) will be required.11
This is intended as an illustration of how the probabilistic forecasting information could be used. In reality the choice of the threshold probability will depend not just on the acceptable number of false alarms and missed events for a particular location but also on a consideration of the water resource zone as a whole together with many other factors, such as statutory regulations and the low-likelihood/high-consequence nature of the resource management problem.
Of course, many of the simplifications adopted here for the purpose of modeling are not valid in reality. With respect to the system analyzed, there are many more considerations across the whole resource zone that are not accounted for in this analysis of reservoirs A and B. With respect to the forecasts’ attributes, lead time, rainfall amounts (for instance, drier than average vs precipitation in the bottom tercile of the climatological distribution), and level of confidence are all important to the decision and are interpreted and taken into account by the human decision-maker, who has extensive local knowledge and experience of the behavior of the water resources system developed over time. Water managers explained that they would need confidence in the rainfall forecast at lead times of a few weeks or months to be confident that drought powers would not be needed, as one week of higher than average rainfall will not guarantee the end of a potential drought event, and most of the drought management actions require more than a week to implement. Moreover, probabilistic forecasts of the particular amount of rain that has the potential to end a drought event will be required: a forecast of drier-than-average weather conditions as used in our example would not provide the required information. Finally, our discussions with water managers indicated that given the low rainfall observed in May and June 2010, the spread of the box plots in Figs. 3 and 4 would not give the water managers enough confidence to make a decision. However, water managers explained that they do not currently know the level of confidence they would need or how they would gain this confidence in a forecast.
4. Comments on the usability of seasonal forecasts for water management
In our discussion with the water managers and regulators to define a modeling case study to investigate the use of probabilistic forecasts in a real decision-making context, we found an openness to explore forecasts that could support decisions on the medium to extended range (from days up to a month). As discussed above, because of the physical characteristics of this catchment, there is potential for using these forecasts if skillful.
The situation was somewhat different when considering longer-range seasonal forecasts (1–7 months out). To assess the risk of failing to meet demand, the water managers currently use a tool called Droughtwatch, which is jointly owned by the water company and EA. Droughtwatch generates scenarios for the reservoir level for the following 12 months, assuming a series of return periods for the inflows that include the minimum historical flow (see Fig. 5; Day 1985). These scenarios could, in principle, be replaced by an ensemble forecast of inflows (obtained by feeding the ensemble of weather forecasts into the hydrological models) that could be used to estimate the probability of crossing the different triggers. While these probabilistic seasonal forecasts of reservoir inflows could in principle be incorporated into the current decision-making framework without significant changes, we found that the openness of the water managers to exploring forecast use did not extend to seasonal forecasts.
In interviews with water managers from the company we worked with on the analysis above and from another water company based in southern England, we identified some of the factors that underlie the reluctance to use seasonal forecasts. One factor was a stated lack of confidence in the quality of seasonal forecasts. Interviewees cited the Met Office’s seasonal forecast for the summer of 2009, which was widely interpreted in the media as a failure (Eden 2011). Some water managers expressed doubt that the trajectory of the drought event of 2012 (which affected central, eastern, and southern England and Wales) could have been predicted by the Met Office and suggested that the Met Office itself had little confidence in its forecasts beyond 5–10 days lead time. Indeed, the Met Office’s current guidance for contingency planners explicitly discourages the use of 3-month outlooks for implementing immediate actions or committing resources (http://www.metoffice.gov.uk/publicsector/contingency-planners). For the case cited by the interviewees, the lack of confidence in the quality of forecasts is, in part, due to a problem of communication: the Met Office’s seasonal forecast for the summer of 2009 was probabilistic, but this was not always clear in media reports.
While the quality and availability of seasonal probabilistic forecasts has increased in recent years in Europe (Doblas-Reyes et al. 2013), the current capability to forecast European climate, and in particular precipitation, at monthly to interannual time scales is hindered by limited potential predictability in the extratropics and the inability of forecasting models to represent some of the key drivers of droughts (Boer and Lambert 2008; Scaife et al. 2010; Zappa et al. 2013). Developing skillful, statistically reliable, and decision-relevant (hydro)meteorological forecasts of the drivers of droughts in the United Kingdom on monthly to seasonal time scales is a problem that is currently the subject of a large research-council funded project [Improving Predictions of Drought for User Decision Making (IMPETUS)] as part of a larger program of research into U.K. droughts and water scarcity (Prudhomme et al. 2015).12
Water managers’ risk aversion also constrains the use of seasonal forecasts. Water managers characterized themselves as very risk averse, specifically in relation to missing an event [i.e., failing to take timely actions for either supply or demand management ahead of a (potential) drought] and thus failing to fulfil their statutory requirements. Some mentioned the Yorkshire drought of 1995, after which Yorkshire Water Services was criticized for mismanagement by two independent inquiries and fined for having brought the region to the verge of water supply failure including through underinvestment (Bakker 2000).
As noted above, the water managers in the northwest linked their aversion to the risk of supply failure to the need to justify management decisions using information acceptable to the regulator. This meant not taking risks based on forecasts. The current regulatory framework13 requires that drought planning must account for at least a repeat of the worst historical drought and ideally for drought events of longer duration and lower rainfall than the historical ones.14 Water managers explained that to avoid their company being subject to an inquiry, they have to demonstrate diligence and responsibility by “planning for the worst”: given the situation now, what if there was a repeat of the rainfall pattern from the worst event on record, and what would they need to do? While this does not only apply to seasonal forecasting time scales, managers described the consequences of a “wrong decision” increasing with the lead time of a forecast.
Given the multiple factors affecting the usability of seasonal forecasts, it is difficult to know whether the main limitation for the uptake of seasonal forecasts is the lack of technical quality. However, our research suggests that even if the forecasts were technically skillful and reliable, the current regulatory and institutional arrangements for water management in the United Kingdom would not incentivize the use of probabilistic seasonal forecasts for drought management. The institutional factors exemplified by water managers’ risk aversion and the constraints imposed by regulatory frameworks recall some of those identified by Rayner et al. (2005) in their study of U.S. water managers’ reluctance to use NOAA’s new seasonal forecasts in the 1990s and by Soares and Dessai (2016) in their study of the uptake of seasonal forecasts in a range of sectors across Europe.
5. Discussion and conclusions
In principle, the additional information about forecast uncertainty that is contained in a skillful probabilistic forecast provides an opportunity to take this uncertainty into account when weighing up the potential costs and benefits of a decision by linking the decision to act with the forecast probability. For example, we have shown that for our model forecasting system, definition of extreme event (drier-than-average conditions), and by optimizing the choice of probability threshold, the manager can hypothetically reduce the expected number of false alarms and missed events by using the probabilistic ensemble instead of a deterministic forecast. For the system in question, the probabilistic forecast information could potentially enable the water company to minimize the number of drought permit/order applications (or use restrictions) that must be mobilized early but are with hindsight unnecessary. However, this only works as expected averages. In practice, given the very high stakes of missing an event, the water manager will require a very high level of confidence of forecasts for a range of rainfall amounts before being willing to use probabilistic forecast information to take a more “risky” decision (including the decision to defer an action) than one based on the conventional information; they would also need to convince the regulator (EA) of the rationale for diverging from management based on historical information.
a. Forecast attributes
The geophysical characteristics of the region and resource zones managed by the water companies have an influence on the types of forecast information that may be of use in different areas. For example, the critical period in the resource zone we have studied is relatively short compared with the southeast of England, where multiple-season events are the main risk. Therefore, medium- to extended-range forecasts with lead times from 1 to 5 weeks have the potential to inform some management decisions in this northwestern catchment because of its flashy characteristics.
However, there appear to be two decision-making contexts for the water managers: “normal” operations in which medium- to extended-range forecasts may provide information that can be used to inform the management of the system and potential drought events, where water managers stated that they would require more reliable and accurate forecasts of particular rainfall amounts for weeks to months ahead to make decisions for the next 6–12 months.
Our analysis of the available ensemble precipitation forecasts for 2010 shows that they are not skillful when verified with station data (as measured by the CRPS). After correcting the mismatch between station data and weather model resolution, precipitation forecasts show skill for week 1 lead time when compared to climatological information. That is to say, beyond week 1, the decision-maker is not better off using the weather forecasts than making decisions based on climatological information. At longer time scales, developing skillful, statistically reliable meteorological forecasts of the drivers of droughts in the United Kingdom on monthly to seasonal time scales is the subject of ongoing research (Prudhomme et al. 2015).
b. Risk and regulation
For the water manager, an important question is whether, at the trigger point where she considers taking action, she can confidently override the “set piece” action (based on the triggers defined in the drought plan) using weather forecasts. If the water manager and, importantly, the regulator have low confidence in the forecast, such a divergence from the plan could have regulatory implications, for example, if the water manager takes a risk-prone decision to defer action and then needs to apply to the EA for drought permits (these could be refused, leaving the water company at risk of failing in its statutory duties or having to bear the costs of finding alternative supplies). The high stakes and extensive economic and reputational costs of missed events for water companies mean that it is unlikely that numerical weather forecasts can be used in this way at present.
The social and institutional factors that influence what kind of information can justifiably be used both for operational management and for broader regulatory frameworks and planning reflect a negotiation of commitments to different priorities among water companies, statutory regulators (environmental, economic, and quality), customers, company boards, investors, and other stakeholders. Such commitments include maintaining acceptable levels of service, protecting the environment, controlling prices, avoiding regulatory sanctions, and answering to shareholders. Lange and Cook (2015) describe how different public and private actors negotiate authority and the exercise of power through (interpretation of) legal criteria; regulatory tools such as drought plans, permits, and orders; and public discourses. They argue that the complexity of information and the need for water companies to defend their decisions and win the cooperation of different stakeholders in this “governance space” causes water companies to focus on reputation and creating confidence in their activities. Our study suggests that this need to base decisions on information that will be credible to different stakeholders may disincentivize the use of uncertain/probabilistic forecasts as opposed to approved procedures (see also Rayner et al. 2005, p. 212).15
We suggest that any steps toward changing the use of information for drought management would involve transparent discussion of stakes, priorities, and capacities. Kirchhoff et al. (2013) have argued that the use of weather and climate information for water management can be influenced by the nature of interactions among physical and social scientists, decision-makers, and stakeholders in organizations such as the U.S. Regional Integrated Sciences and Assessments (RISA); examples of the use of forecasts to inform decisions in other sectors include Public Health England’s heat wave plan and cold weather plan,16 the development of which involves regular cross-sector discussions and periodic evaluations. As Broad et al. (2007) argue, using a case from Brazil, the value of forecast information for water management changes according to societal shifts, including changes in water demand and governance (who has a say in setting rules and allocations; what are their priorities and risk perceptions?).
The reluctance to use forecasts appears to be important in drought management; however on the basis of our case study, we cannot generalize to all aspects of operation. For example, water managers noted that they can do cost optimization when the resource is “healthy” (e.g., using weather forecasts for pumping decisions during the wet season).
c. Challenges in operationalizing weather forecasts
Using weather forecasts within water resources models to forecast water supply involves two components: the forecasts and the transfer to the hydrological/system assessment. The water managers commented that making the connections necessary for regional modeling with weather forecasts is a big step and one that will only be justifiable as an investment when these forecasts improve. However, they did not specify how this improvement should be measured or what would be seen as a “good enough” forecast for this investment.
Water managers are used to considering probabilistic information: for example, return period inflows currently used by many water companies are probabilistic. But our study suggests that, while probabilistic forecasts could in principle be used to minimize false alarms and thus increase the efficiency of drought management, the stakes of missed events are too high for the managers in this study to be willing to use forecasts in this way. The regulatory framework for drought management of public water supply places water managers’ decisions in the context of a plan that requires them to take into account the possibility of a repeat of the worst conditions on record and in which a failure of service that is not deemed to have been caused by unprecedented environmental conditions is seen as an “absolute” failure. For water managers, taking risks based on uncertain forecasts—even if the uncertainty skillfully represents the chaotic nature of the weather system—puts them at risk of their worst-case scenario (a failure of service). In this setting, water managers are unlikely to be willing to use probabilistic weather forecasts as more than contextual information. An important consideration is the requirement by water managers for more confidence in the forecasts. This can be understood as a request for very sharp and accurate forecasts (to the extent that they are effectively deterministic and perfect). Such a requirement cannot be satisfied: the point of ensemble systems is to quantify uncertainty, and even if a perfect ensemble forecasting system was possible, uncertainty will always be present because of the chaotic nature of the climate system.
Assuming the forecasting system is skillful and reliable (i.e., assuming the ensemble encompasses the real-world uncertainty), an approach consistent with the current regulatory framework could involve taking into account the worst forecasted scenario (i.e., the driest ensemble member once the weather forecast has been run through the hydrological models), as well as worst recorded cases. This might not be optimal (in the sense of optimizing the fraction of actions in vain for instance) but would have the advantage of incorporating in the estimation of “worst case” dynamic information that takes into account changes in the climate system. With increased acknowledgment in the water sector of nonstationarity and the need to look beyond historical records (Milly et al. 2008), probabilistic weather forecasts may be a part of a suite of technical and nontechnical changes seen in the water sector to address the challenges of water security.
We are grateful to the Oxford Martin School for funding this study as part of the Oxford Martin Programme on Resource Stewardship (authors 1 and 2) and to the Natural Environment Research Council (NERC)/Research Councils U.K. for support under the IMPETUS project NE/L01047X/1 (author 2). We gratefully acknowledge the generous assistance of all interviewees, including at United Utilities and the Environment Agency, and thank ECMWF for facilitating access to forecasts used in the analysis. Many thanks to Steve Rayner, Tim Palmer, Elisabeth Stephens, Fai Fung, Edoardo Borgomeo, and David Richardson for insightful discussions and helpful information during the development of this work and to editors and reviewers at Weather, Climate, and Society for their helpful comments on earlier versions of this article. Because of the confidential nature of some of the research materials supporting this publication, not all of the data can be made accessible to other researchers. Please contact the authors for more information.
Supplemental information related to this paper is available at the Journals Online website: https://doi.org/10.1175/WCAS-D-16-0072.s1.
The EA is the environmental regulator in England. While water companies are also regulated by other bodies, for example for pricing and drinking water quality, the EA has the responsibility for “safeguarding the environment during drought and overseeing the actions water companies take to secure public water supplies” (Environment Agency 2015).
A water resource zone is defined in national guidelines as “an area within which the management of supply and demand is largely self-contained (apart from agreed bulk transfers of water). Within the WRZ [water resource zone], supply infrastructure and demand centers are generally integrated to the extent that customers in the WRZ should experience the same risk of supply failure. Consequently all customers share the same level of service” (Environment Agency et al. 2012, 18–19).
A 2-season drought event describes two consecutive dry winters or summers, for instance.
The following descriptions are taken from the Final Drought Plan (United Utilities Water 2014). Drought permits, granted by the EA to a water company, authorize the water company to “take water” from specified sources or modify or suspend restrictions or obligations to which the water company is subject. The drought plan uses the term “drought permit/order” to differentiate these from drought orders for nonessential use. Drought orders are granted by the Secretary of State to water companies or the EA. Ordinary drought orders can include the same powers to abstract water as drought permits, but they can also authorize water companies to take other actions. An emergency drought order gives water companies complete discretion on the uses of water that may be prohibited or limited; they can authorize use of standpipes, water tanks, or rota cuts.
Water industry legislation introduced in 1991 allowed water companies to restrict the use of hosepipes (or similar apparatus) for private use, commonly known as a hosepipe ban. New legislation introduced in 2010 gives water companies further powers to restrict water use by customers, and the current drought plan refers to “water use restrictions” rather than hosepipe bans (United Utilities Water 2014, p. 77).
The water company is currently using weather forecasts to forecast demand, but these are limited to 1–2 weeks.
We do not include the estimates of the sampling uncertainty here because our aim is to illustrate how the probabilistic information could be used within the current decision-making framework and not to provide robust estimates of false positives and false negatives.
Author 2 has been part of the IMPETUS project.
By this we mean the laws, guidelines, tools, and institutional remits that shape drought management and response, in this case, particularly for the water company.
Recent updates to the EA’s drought planning guidelines (https://www.gov.uk/guidance/write-a-drought-plan) strongly encourage water companies to plan to provide supplies through droughts more severe than those on record. This might involve applying projections or stochastic modeling to account for climatic uncertainties associated with climate change.
See, for example, https://www.gov.uk/government/publications/heatwave-plan-for-england.