1. Introduction
Part I of this two-part paper (McCollor and Stull 2008) documents the meteorological evaluation of a short-range ensemble forecast (SREF) system for hydrometeorological applications in complex terrain. A standardized set of evaluation methods, scores, and diagrams (as described in Hamill et al. 2000) was employed to compare and judge the meteorological skill of the SREF system. However, skill and dispersion measures of the meteorology alone do not give a complete assessment of SREF forecast values (Anderson-Berry et al. 2004).
Complete evaluation of the benefit of a forecast system should consider the weather sensitivity and decision-making processes of a particular end user or group of users. Societal and environmental impacts are also important components of weather forecast evaluation; however, in most cases economic impacts are easier to directly identify and quantify.
In a study by Roulston et al. (2006), laboratory experiments documented the ability of nonspecialist weather forecast users to make sound decisions. That study showed how users increase their expected economic reward while reducing their exposure to risk by incorporating quantitative estimates of weather forecast uncertainty into their decisions.
We employ economic skill measures to evaluate user-dependant hydrometeorological forecasts. Two economic models adapted for the hydroelectric energy sector are examined in this paper: a cost–loss ratio decision model and a utility-maximizing decision theory model. The economic models are applied in an operations assessment at Jordan River, a hydroelectric facility on the southwest coast of Vancouver Island, British Columbia, Canada. Figure 1 includes a photo of the Jordan River site plus three other reservoir facilities in this region of complex terrain.
Forecast evaluation using economic models applicable to the hydroelectric sector is introduced in section 2. Application of the models to probabilistic forecasts from an SREF system is given in section 3. A case study application is presented in section 4, and conclusions are provided in section 5.
2. Forecast evaluation via economic models
a. Background
In rapid runoff response pluvial watersheds (the subject of this study), foreknowledge of inflow allows optimal reservoir operation under a set of given operating constraints. Operating constraints may include, for example, optimizing the depth (hydraulic pressure head) to maximize electrical generation, avoiding spill under high-inflow events, maintaining minimum flow requirements under dry conditions to provide adequate water for fish survival and aquatic habitat, and maintaining stationary reservoir levels under fluctuating inflow conditions to enhance recreational usage.
Uncertainty in future rainfall, hence uncertainty in future reservoir inflow, poses significant problems for reservoir operators tasked with meeting stated operating constraints. Minimizing future inflow uncertainty through the knowledgeable use of probabilistic forecasts can be analyzed with suitable economic models.
b. Cost–loss model
Foreknowledge of high inflow allows a reservoir operator to draft (i.e., lower the water level of) a reservoir ahead of the inflow event. This allows the operator to obtain the full value of the water by routing all reservoir water through the turbines and generators before and during the event. Thus, the operator can capture all of the precipitation-induced runoff without being forced to release excess water (and bypass the revenue-producing generators) downstream. This action is colloquially known as “digging a hole in the reservoir” to capture subsequent inflow. In the managed watersheds analyzed in this study, the dominant source of higher inflow is rainfall events produced by landfalling Pacific frontal systems impinging on the mountainous coastal terrain of southwest British Columbia (see Fig. 1 in Part I for a reference map of the region).
There is an economic risk associated with operating a reservoir in the manner outlined above. If no action is taken prior to a high-inflow event, the excess water flowing into the reservoir is released or spilled without generating revenue, hence incurring an economic loss from the value of that water. If the operator does in fact generate power in advance of the high-inflow event, an economic cost is entailed involving hydraulic head loss from lowering the reservoir elevation, which may not be offset if the forecast inflow event does not occur.
Economic forecast value can be addressed using the concept of a simple (static) cost–loss ratio decision model (Murphy 1977; Katz and Murphy 1997; Richardson 2000; Zhu et al. 2002; Richardson 2003). Sensible weather forecast examples employing a cost–loss ratio decision model in specific weather forecast evaluations are provided in the literature for road weather forecasts (Thornes and Stephenson 2001), air quality forecasts (Pagowski and Grell 2006), severe weather forecasts (Legg and Mylne 2004), precipitation forecasts from a global ensemble prediction system (Mullen and Buizza 2002), precipitation forecasts over the southwestern United States (Yuan et al. 2005), temperature forecasts for the energy sector (Stensrud and Yussouf 2003), and medium-range flood predictions for watersheds in low-relief terrain (Roulin 2007). A description of the cost–loss model as applied here to reservoir operations is provided below.
A reservoir operator has essentially two alternative courses of action to choose from: either draft the reservoir or not. The choice depends partly on the capacity of the reservoir to hold additional water, and largely on the precipitation forecast, since precipitation is highly correlated with inflow in these particular watersheds (see Fig. 2 for an example of precipitation–inflow correlation). Each action has an associated cost (from head loss) and leads to an economic benefit (minimize lost revenue due to forced spill) depending on the precipitation (hence inflow) that occurs. The task of the reservoir operator is to choose the appropriate action in each instance that will minimize both the cost and the expected loss over the long term.
The cost–loss model is especially useful in this study because it shows how probabilistic forecasts can be used in the decision-making process, and provides a measure of the benefit of probabilistic forecasts over deterministic or ensemble-average forecasts. In the case at hand, the forecast user (the reservoir operator) is sensitive to the specific daily precipitation forecast thresholds analyzed in the companion paper: 5, 10, 25, and 50 mm day−1.
The four possible combinations of action and occurrence (with the net cost depending on what happened and what action was taken) are summarized in Table 1. A separate analysis will be performed for each precipitation threshold. In Table 1, a is the number of hits (successful event forecasts), b is the number of false alarms (forecast but not observed), c is the number of misses (observed but not forecast), and d refers to correct rejections (neither forecast nor observed). Hits and false alarms both incur a cost of taking preventative action (C), since the operator drafts the reservoir whenever the event is forecast. Misses are associated with a loss due to the lack of prevention (L). Correct rejections incur no expenses either way. The reservoir operator acts to minimize the average long-term loss by taking appropriate action on each occasion.
For a particular SREF system and a given precipitation threshold, the value V of the forecasts is solely determined by the user’s cost–loss ratio. By plotting V as a function of cost–loss ratio, r, a water resource manager can determine if the full ensemble forecast system provides more value than does a deterministic forecast or the ensemble mean forecast.
For a reliable ensemble forecast system, the maximum value for a particular user will be obtained by choosing a forecast probability equal to the user’s particular cost–loss ratio, r (Richardson 2000, 2003). Also, the maximum value of V always occurs when r =
c. Decision theory model
The cost–loss model described in section 2b is a binary-decision model. Other techniques that incorporate economic decision making for continuous input–output variables (Katz and Murphy 1997; Smith et al. 2001) such as precipitation and power production can also be applied to reservoir planning processes.
In certain situations a reservoir operator may be tasked with maintaining a constant-level reservoir under variable inflow conditions. Optimal use of probabilistic forecasts may be analyzed using a maximum-utility decision theory economic model. In decision theory models, optimal decisions in uncertain environments are associated with the maximum of expected rewards.
Formal decision theory was originally developed in the first half of the twentieth century by mathematicians aiming at modeling the process by which optimal rational decisions are made (Baldi 2001). In this model, a decision maker in an uncertain environment is faced with a choice of possible actions. Each action has an associated consequence. The consequences are not known with certainty but are expressed as a probability of occurrence. The optimal decision-making strategy is to maximize the expected value (i.e., maximize gain or minimize loss), where expected value refers to a weighted average taken over all possible probabilistic states of the environment. Thus, decision theory naturally leads to probabilistic determination, and an ensemble forecast is a method of defining such weights or probabilities.
The expected utility hypothesis in economics (Laffont 1989) states that the utility of a user facing uncertainty is calculated by considering the utility in each possible state and constructing a weighted average, the weights being the user’s estimate of the probability of each state. For hydrometeorological analysis, the user must define a utility function that describes the benefit or loss incurred based on a particular reservoir operation decision. The uncertainty in the decision is derived from uncertainty in future rainfall, and the probability of such an uncertain rainfall event is determined from the SREF prediction system.
The utility function for this hydrologic model is adapted from another renewable energy source (wind generation) utility function as described by Smith et al. (2001). Short-term rainfall, hence reservoir inflow, is a major factor in determining the capacity to supply electrical energy to the grid. The reservoir operator must be able to forecast generation in advance, subject to maintaining a constant-level reservoir (for the benefit of recreational and other users), since failure to provide energy specified in a contract will lead to purchasing energy at market prices to meet a prespecified contractual delivery.
3. Results
a. Cost–loss model
The forecast user (in this case a reservoir manager) is faced with the question of how high the forecast probability of a particular event needs to be before the threat is great enough to warrant taking preventative action. The decision maker needs to set a threshold probability, pt, and take preventative action only when the forecast probability exceeds pt. By varying pt over 0/M, 1/M, 2/M, . . . 1, where M is the number of ensembles employed in the SREF, a sequence of hit H versus false alarm F rates can be plotted that traces out the ROC curve for the SREF system (see section 4c in Part I). Different value traces, V, are then plotted as a function of the C/L ratio [see Eq. (4) for each distinct point (F, H) corresponding to each particular pt on the ROC curve]. The value curves for each precipitation threshold are displayed in Fig. 3 for the day-1 forecasts and in Fig. 4 for the day-2 forecasts.
In each of the individual V versus C/L ratio graphs shown in Figs. 3 and 4, the envelope curve (displayed with a heavy solid line) shows the optimum value of the SREF system, obtained when each decision maker employs the probability threshold pt that maximizes V for their specific C/L ratio. As seen in Figs. 3 and 4, the envelope curve is never less than any of the individual pt curves, indicating that no particular single forecast will benefit different users utilizing different C/L ratios. The maximum value (Vmax) of the envelope curve for V occurs at that value of C/L equal to the climatological base rate
The shape of the Vmax envelope curves for the day-1 forecasts are similar to those for the day-2 forecasts, for all four precipitation thresholds. This indicates little change in relative value among different users employing different C/L ratios in their decision making. However, the Vmax envelope curve is shifted downward slightly in the day-2 forecasts compared to the day-1 forecasts, indicating that, as expected, the measured value of the forecasts decreases as the forecast range is extended. If the probabilistic forecasts were extended into the midrange (3–10-day forecasts), the Vmax envelope curve would necessarily collapse as forecast skill approaches climatology [Eq. (1)]. A similar assessment for midrange forecasts (a potential avenue for future research) would indicate how far into the future these forecasts maintain their positive economic value.
Plotting the envelope Vmax versus C/L allows a clearer view of how different forecast users benefit from the probabilistic nature of SREF systems, especially when plots of deterministic forecasts and ensemble-average forecasts are included for comparison. In Figs. 5 and 6 (for day-1 and day-2 forecasts, respectively), the maximum potential value curves are plotted along with a best deterministic forecast and an ensemble-average forecast. The best deterministic forecast was chosen among the individual SREF members as the one with the best error characteristics (see section 4a in Part I), namely the very high-resolution MM5–4 km, model. The ensemble-average forecast is the equally weighted average of all SREF system member forecasts. The ensemble-average forecast is included for comparison, although in practice caution is advised in using this forecast because it may not represent a physically realizable solution to the model equations (in the manner that the average of the numbers on a six-sided die, 3.5, is not a physically realizable outcome of rolling the die).
The plots in Figs. 5 and 6 indicate that the SREF system provides the best economic value for both day-1 and day-2 forecasts, all precipitation thresholds, and all C/L ratios. Also, these figures show positive economic value to all users at the highest precipitation threshold of 50 mm day−1. However, for lower precipitation thresholds, reservoir managers employing the very lowest C/L ratios progressively lose value as the precipitation threshold lowers from 25 to 10 to 5 mm day−1. The advantage of a probabilistic forecast system is evident from the fact that the EPS exhibits a larger range of forecast users that can economically benefit from the forecasts than either the deterministic forecasts or the ensemble-average forecasts (namely, the dotted and dashed curves are narrower and lower than the ensemble envelope curve).
For the higher precipitation thresholds of 25 and 50 mm day−1, the ensemble-average forecast outperforms the deterministic forecast in terms of value; however, the probabilistic forecasts are even more advantageous as they extend the valuable range of the forecasts to a broader cast of users. At the lowest precipitation threshold of 5 mm day−1, the measured maximum values Vmax of the SREF system, the deterministic forecast, and the ensemble-average forecast are the same. As the thresholds and their impacts on the reservoir operations increase, the measured maximum value Vmax of the SREF system becomes greater than that of the deterministic and ensemble-average forecasts.
Information about choosing the correct probability threshold pt for each decision maker can be shown by plotting the value V as a function of pt for different C/L ratios (see Fig. 7 for the day-1 forecasts and Fig. 8 for the day-2 forecasts). The plots show that reservoir managers with low C/L = 0.2 who have low costs but potentially high losses benefit from taking action (pregenerating electricity and thus lowering the reservoir in advance of a forecast precipitation event) at a precipitation threshold exceedance probability as low as 10%.
As the C/L ratio increases to 0.6 for specific reservoirs, the managers should wait until the probability of an event is 30% before taking preventative action. For managers of a reservoir with C/L = 0.8, the associated high cost of lowering the reservoir in advance of a forecast inflow event and limited potential losses from the forced release or spill mean that they should wait until forecasts are in the 70%–80% range before pregenerating.
Day-1 SREF system forecasts are measurably better than day-2 forecasts. The plots in Figs. 7 and 8 indicate the trend that the probability of an event in the day-2 forecast time frame should be about 10% higher than the probability in the equivalent day-1 forecast to ensure that the same value ensues from action taken. In other words, reservoir managers operating a reservoir with C/L = 0.2 could pregenerate in the day-2 time frame when the precipitation event forecast probability reaches 20%. Managers operating a reservoir with C/L = 0.8 can take preventative action 2 days before a forecast event of 5 or 10 mm day−1, but the probability of occurrence must be close to 90%. This effect is less evident for 25 mm day−1 precipitation events and is nonevident for 50 mm day−1 events, likely because the smaller number of cases for these climatologically rarer events introduces sampling errors into the analysis (as shown in section 4b in Part I).
Looking more closely at the relation between precipitation threshold and value for different SREF combinations, Figs. 9 and 10 show this relation for the day-1 forecasts and day-2 forecasts, respectively. The full-ensemble SREF provides value for a greater range of users at all precipitation thresholds, compared to the deterministic forecast and the ensemble-average forecast. As the precipitation threshold increases, the forecasts provide value to more and more users toward lower C/L values.
There are specific links between the economic value analysis of the cost–loss model and the ROC analysis presented in section 4c in Part I. Figure 11 shows the ROC curve for the day-1 forecasts with a 10 mm day−1 threshold alongside the corresponding maximum value envelope curve. The deterministic high-resolution forecast and ensemble-average forecast are each represented by single points in the ROC diagram. These points are shown connected with straight-line segments to the upper-right and lower-left corners of the ROC diagram.
The deterministic forecast point and the ensemble-average forecast point both lie near the probability forecast ROC curve. This suggests that basic forecast system performance is similar for all three forecast systems. The benefit of the probability forecast depends on the varying needs of different users. As seen on the value curve, all three forecast systems show similar value for a narrow range of users with C/L ratios between about 0.3 and 0.4. For C/L values outside this range, the ensemble forecasts prove most economically valuable.
An example using ensemble precipitation forecasts in a cost–loss model for specific reservoirs is given in section 4a(1).
b. Decision theory model
The utility functions calculated from Eq. (7) for Jordan River operations indicate the variation of utility with pricing structure. The particular utility function explored here displays increasing penalties as the Sm/Sc ratio increases, and increasing benefits as higher energy contract amounts are selected. The penalties and benefits increase jointly with the precipitation threshold indicated.
An example using ensemble precipitation forecasts to maximize the expected value function for Jordan River is given in section 4b.
4. Case studies
a. Cost–loss model
1) Model development
In this section we provide a practical example of the cost–loss model evaluation for specific watersheds in southwest British Columbia. The case study represents a simplification of reservoir management and operational constraints, and is intended to examine the value of probabilistic precipitation forecasts for inflow into reservoirs.
P = generator power output (W),
η = turbine/generator efficiency as a fraction,
γ = specific weight of water = 9807 Nm−3 at 5°C,
Qt = flow through the turbine (m3 s−1), and
H = difference in elevation (termed “head”) between the reservoir level and turbine (m).
Electric energy is typically bought and sold in units of either kilowatt hours (kWh) or megawatt hours (MWh), and the price of electrical energy is typically quoted in dollars per MWh. The conversion factor between joules and MWh is 1 MWh = 3.6 × 109 J.
We will use the following general variables to describe the hydrometeorological forecast inflow model (we use a time increment of 1 day for all measurements, calculations, and forecasts):
h2 = the difference between the lowered reservoir elevation and the low-level intake (m),
h3 = the difference between the reservoir at full operating level and the lowered reservoir elevation prior to an expected inflow event (m), and
Qs = the water that spills past the generator (m3) without producing any power (hence any revenue).
We will also use the following reservoir-specific constants in describing the model:
h1 = nominal head [the height difference between the low-level intake and the turbine (m)],
Qb = the daily base inflow (m3),
Ar = the surface area of the reservoir (m2), and
G = the space in the reservoir available for storage (m3) = (h2 + h3) × Ar.
When the reservoir is full, H = h1 + h2 + h3. If the reservoir is lowered an amount h3 in anticipation of a rainfall-induced inflow event, the overall head H will be reduced to H = h1 + h2, resulting in less power production.
The cost–loss ratio for a particular reservoir is determined in the following way. The loss that would ensue if the reservoir was full during an inflow event is the value of the water Qs that spills past the generator without producing any power (hence any revenue).
The C/L ratio can then be calculated for individual reservoirs. See Table 2 for the physical parameters of the reservoirs included in the study, and Table 3 for the associated ratios for this basic C/L model.
Equation (15) shows that small C/L ratios are associated with reservoirs with large surface areas, high head, large storage, and low baseflow. This type of reservoir is typically found in regions of high mountains and deep valleys where most of the reservoir inflow stems from precipitation-generated surface runoff (climatically high-rainfall zones). Conversely, large C/L ratios are associated with reservoirs with small surface areas, low head, low storage, and high baseflow. This type of reservoir would typically be found in areas of low, rolling relief where much of the reservoir inflow derives from subsurface aquifers (climatically low-rainfall zones). The reservoirs included in this study are of the former subtype and are characterized by very low C/L ratios.
Table 3 also lists the C/L ratios for individual reservoirs (for precipitation thresholds of 50 mm day−1) incorporating the climatological frequency of the event [Eq. (16)] and the variable price of energy [Eq. (17)].
2) Example
Dollar value estimates for L can be calculated from Eqs. (12) and (13). The resulting cost estimates for C/L operating decisions at Jordan River based on the 50 mm day−1 precipitation threshold are given in Table 4. The full ensemble provides a net benefit (lower cost) of $80,000 ($40,000 annually) or 8% improvement compared to the deterministic forecasts, and a net benefit of $545,000 ($272,500 annually) or 57% improvement compared to the ensemble average forecasts.
b. Decision theory model
In this section we provide a practical example of the decision theory model for a specific watershed in southwest British Columbia. As with the cost–loss model, this case study represents a significant simplification of reservoir operating decisions and is intended to examine the value of probabilistic precipitation forecasts derived from SREF-generated ensembles.
The energy produced by the reservoir outflow, Qout, is given by Eq. (10). If the constraint to maintain a level reservoir is met, then Qout = Qin. The day-1 forecasts of 24-h precipitation and associated daily inflow observations from Jordan River on southwest Vancouver Island were used in this analysis. The linear Pearson correlation coefficient for Jordan River is 0.88 (see Fig. 2). Therefore, the hydrologic response (inflow) to rainfall can be approximated by a linear function: Qin = κR = Qout = Qt.
The results show that when the contract price is equivalent to the market price, there is little difference whether using the full ensemble, the ensemble average, or the deterministic single forecast to provide probability estimates. Note that using climatologic representative probabilities results in a loss for all values of Sm/Sc. However, as the spread between the market price and contract price widens and the Sm/Sc ratio increases, a different profit scenario emerges. The ensemble average forecast provides the greatest profit, followed closely by the full 11-member ensemble. The profit realized by using the deterministic high-resolution forecast drops quickly relative to the SREF forecasts.
5. Conclusions
Weather forecasts are made and used under the assumption that forecast users can take preventative action, prior to a predicted weather event, that provides more benefit to the user than having no forecast at all. This assumption should be specifically tested to aid weather forecast providers in improving their forecasts, and to assist weather forecast users in assessing the benefit of the forecasts to them.
Forecasts can be tested by assessing both their meteorological skill and their economic value. Forecast skill is generally determined by assessing the accuracy of a forecast-producing system relative to a given standard such as climatology, persistence, or another forecasting system. Forecast skill will vary among different forecast-producing systems, but remains user independent. Part I assessed the meteorological skill of a short-range ensemble forecast system designed to provide 1- and 2-day precipitation forecasts in complex mountainous and coastal terrain.
Forecast value is a separate assessment of a forecasting system that takes into account diverse needs and benefits attributed to different forecast users. This paper comprises an economic value assessment of the same set of forecasts that provided the basis for the meteorological skill assessment described in Part I.
The particular forecast users studied here are water resource managers tasked with making efficient and effective use of rainfall that ultimately drives hydroelectric power in rainfall-fed mountain–coastal reservoirs.
Employing a static cost–loss economic model adapted to reservoir operations for hydroelectric managers, a forecast system that provides a suite of ensembles yields positive economic value for a broader range of users than do either a single fine-resolution forecast model or a single forecast composed of the ensemble average. Second, the study showed that different forecast users, characterized by specific cost–loss ratios, require different precipitation probability thresholds to trigger preventative actions. Users with different cost–loss ratios correspond to operators of reservoirs with different head-loss parameters, different sizes of reservoirs, and different baseflows (e.g., lower C/L ratio applies to reservoirs with low base inflows, high head, large areal reservoirs, and high storage, all of which are typical of rainfall-fed mountain watersheds). The larger economic benefit of probabilistic forecasts is achieved by allowing users to match forecast probabilities with cost–loss ratios specific to their particular operational needs.
Decision theory, employing a utility function designed specifically for market selling of hydroelectric energy from rapid response rainfall-dominated reservoirs, was used to quantify the positive impact (profit generation) from employing an EPS to generate probabilistic forecasts. The forecasts generated by the SREF system analyzed in Part I are superior to a single, high-resolution deterministic forecast in maximizing profit from the particular hydroelectric facility (Jordan River) studied here, with the ensemble-averaged forecasts outperforming the full suite of ensembles by a relatively small margin.
Employing economic models such as a static binary cost–loss model, or a continuous decision theory model, is a preliminary step in assessing the value of ensemble-based precipitation forecasts to water resource managers. The ultimate goal of ensemble forecasting for reservoir operating planners is to be able to translate the uncertainty in future weather into uncertainty in runoff and inflow, and ultimately the amount of energy generated at a particular reservoir/hydroelectric facility. The uncertainty in overall inflow must also incorporate uncertainty in the hydrologic model that translates forecast precipitation and temperature into reservoir inflow. Future research will include a hydrometeorological model driven by probabilistic meteorological input variables to determine inflows and, ultimately, available energy.
Acknowledgments
The authors thank BC Hydro Corporation and the Meteorological Service of Canada for providing the precipitation observations necessary to perform this study. In addition, the Geophysical Disaster Computational Fluid Dynamics Centre (GDCFDC), within the Department of Earth and Ocean Sciences at the University of British Columbia (UBC), is acknowledged for providing the numerical model forecasts used in this study. In particular, we thank Mr. George Hicks II for his efforts in making the numerical forecast information available to us. The GDCFDC gratefully acknowledges the following organizations for computer-purchase grants: Canadian Foundation for Innovation, British Columbia Knowledge Development Fund, and UBC. Additional grant support was provided by the Canadian Natural Sciences and Engineering Research Council, and Environment Canada. The authors thank the two anonymous reviewers for their suggestions to improve the paper.
REFERENCES
Anderson-Berry, L., Keenan T. , Bally J. , Pielke R. Jr., Leigh R. , and King D. , 2004: The societal, social, and economic impacts of the World Weather Research Programme Sydney 2000 Forecast Demonstration Project (WWRP S2000 FDP). Wea. Forecasting, 19 , 168–178.
Baldi, P., 2001: The Shattered Self: The End of Natural Evolution. MIT Press, 259 pp.
Gulliver, J. S., 1991: Hydraulic conveyance design. Hydropower Engineering Handbook, J. S. Gulliver and R. E. A. Arndt, Eds., McGraw-Hill, 5.1–5.81.
Hamill, T. M., Mullen S. L. , Snyder C. , Toth Z. , and Baumhefner D. P. , 2000: Ensemble forecasting in the short to medium range: Report from a workshop. Bull. Amer. Meteor. Soc., 81 , 2653–2664.
Katz, R. W., and Murphy A. H. , 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, 222 pp.
Laffont, J-J., 1989: The Economics of Uncertainty and Information. MIT Press, 303 pp.
Legg, T. P., and Mylne K. R. , 2004: Early warnings of severe weather from ensemble forecast information. Wea. Forecasting, 19 , 891–906. Corrigendum. 22 , 216–219.
McCollor, D., and Stull R. , 2008: Hydrometeorological short-range ensemble forecasts in complex terrain. Part I: Meteorological evaluation. Wea. Forecasting, 23 , 533–556.
Mullen, S. L., and Buizza R. , 2002: The impact of horizontal resolution and ensemble size on probabilistic forecasts of precipitation by the ECMWF Ensemble Prediction System. Wea. Forecasting, 17 , 173–191.
Murphy, A. H., 1977: The value of climatological, categorical, and probabilistic forecasts in the cost–loss ratio situation. Mon. Wea. Rev., 105 , 803–816.
Pagowski, M., and Grell G. A. , 2006: Ensemble-based ozone forecasts: Skill and economic value. J. Geophys. Res., 111 .D23S30, doi:10.1029/2006JD007124.
Richardson, D. S., 2000: Skill and relative economic value of the ECMWF Ensemble Prediction System. Quart. J. Roy. Meteor. Soc., 126 , 649–667.
Richardson, D. S., 2003: Economic value and skill. Forecast Verification: A Practitioner’s Guide in Atmospheric Science, I. Jolliffe and D. B. Stephenson, Eds., Wiley, 164–187.
Roulin, E., 2007: Skill and relative economic value of medium-range hydrological ensemble predictions. Hydrol. Earth Syst. Sci., 11 , 725–737.
Roulston, M. S., Bolton G. E. , Kleit A. N. , and Sears-Collins A. L. , 2006: A laboratory study of the benefits of including uncertainty information in weather forecasts. Wea. Forecasting, 21 , 116–122.
Smith, L. A., Roulston M. S. , and von Hardenberg J. , 2001: End to end forecasting: Towards evaluating the economic value of the ensemble prediction system. ECMWF Tech. Rep. 336, 29 pp.
Stensrud, D. J., and Yussouf N. , 2003: Short-range ensemble predictions of 2-m temperature and dewpoint temperature over New England. Mon. Wea. Rev., 131 , 2510–2524.
Thornes, J. E., and Stephenson D. B. , 2001: How to judge the quality and value of weather forecast products. Meteor. Appl., 8 , 307–314.
U.S. Energy Information Administration, 2006: Annual electric power industry report. Form EIA-861.
Yuan, H., Mullen S. , Gao X. , Sorooshian S. , Du J. , and Juang H-M. H. , 2005: Verification of probabilistic quantitative precipitation forecasts over the southwest United States during winter 2002/03 by the RSM ensemble system. Mon. Wea. Rev., 133 , 279–294.
Zhu, Y., Toth Z. , Wobus R. , Richardson D. , and Mylne K. , 2002: The economic value of ensemble-based weather forecasts. Bull. Amer. Meteor. Soc., 83 , 73–83.
Photos of four BC Hydro dams in southwestern BC, Canada. Clockwise from upper left: Jordan Dam, Daisy Dam, Wahleach Dam, and La Joie Dam. Daisy Dam is seen spilling 700 m3 s−1 after very heavy rain in mid-October 2003.
Citation: Weather and Forecasting 23, 4; 10.1175/2007WAF2007064.1
Daily precipitation vs daily inflow for the Jordan River on southwest Vancouver Island. (top) The daily inflow (flow rate in m3 day−1) is determined from reservoir elevation and outflow measurements for the Jordan River reservoir system. (bottom) The daily precipitation (mm day−1) was observed at a precipitation gauge in the middle of the watershed. The period of observations is from April 2003 to March 2005. The Pearson correlation coefficient for these two graphs is 0.88, indicating a high correlation between daily precipitation and daily reservoir inflow. For reservoirs throughout the study region the average Pearson correlation coefficient between daily precipitation and daily inflow is 0.70.
Citation: Weather and Forecasting 23, 4; 10.1175/2007WAF2007064.1
Forecast value as a function of C/L for day-1 forecasts. Precipitation thresholds are as indicated in the figures. The thin curves represent forecast values calculated for different probability thresholds pt. The thin curves begin with the lowest probability threshold, pt = 1/11 (dotted line), and continue through increasing values of pt ending with the highest threshold, pt = 11/11 (dashed line). The heavy solid line shows the envelope curve of optimal value (where each user chooses the probability threshold that maximizes the value for his or her specific cost–loss ratio).
Citation: Weather and Forecasting 23, 4; 10.1175/2007WAF2007064.1
Forecast value as a function of C/L for day-2 forecasts. The information presented is in the same form as in Fig. 3.
Citation: Weather and Forecasting 23, 4; 10.1175/2007WAF2007064.1
Forecast value envelope as a function of C/L for day-1 forecasts. Precipitation thresholds are as indicated in the figures. The dashed line is the deterministic high-resolution forecast and the dotted line is the ensemble-average forecast (the abscissa is plotted with a logarithmic axis for C/L to enhance clarity). Since the maximum value occurs at the climate frequency of the threshold event, the curves for the more extreme events are concentrated around lower C/L values.
Citation: Weather and Forecasting 23, 4; 10.1175/2007WAF2007064.1
Forecast value envelope as a function of C/L for day-2 forecasts. The information presented is in the same form as in Fig. 5.
Citation: Weather and Forecasting 23, 4; 10.1175/2007WAF2007064.1
Forecast value as a function of probability threshold for day-1 forecasts. Precipitation thresholds are as indicated in the figures. Different cost–loss ratios are shown as a solid line (C/L = 0.2), dashed line (C/L = 0.4), dotted line (C/L = 0.6), and dashed–dotted line (C/L = 0.8). A value of 1.0 indicates perfect forecasts for users while values below zero indicate that climatological forecasts are more valuable. The graphs show that users with higher C/L ratios must wait for a higher probability forecast of the given event from the suite of ensembles (i.e., the likelihood of the event occurring is more certain) before taking action to receive value from the forecasts.
Citation: Weather and Forecasting 23, 4; 10.1175/2007WAF2007064.1
Forecast value as a function of probability threshold for day-2 forecasts. The information presented is in the same form as in Fig. 7.
Citation: Weather and Forecasting 23, 4; 10.1175/2007WAF2007064.1
Each bar in the figure spans the portion of the C/L range having positive value for different SREF configurations and for different precipitation thresholds for the day-1 forecast. The bars are labeled with the SREF configurations for the full ensemble, the very-high resolution deterministic forecast, and the ensemble-average forecast. Longer bars provide value to a wider range of users.
Citation: Weather and Forecasting 23, 4; 10.1175/2007WAF2007064.1
Each bar in the figure spans the portion of the C/L range having positive value for different SREF configurations and for different precipitation thresholds for the day-2 forecast. The information presented is in the same form as in Fig. 9.
Citation: Weather and Forecasting 23, 4; 10.1175/2007WAF2007064.1
The relationship between ROC and forecast value. Curves are for day-1 forecasts with precipitation thresholds of 10 mm day−1. (left) The ROC curve with the SREF probability threshold pt square markers corresponds to (right) the thick solid envelope value curve. The dashed lines in both figures are the very-high resolution deterministic forecasts and the dotted lines in both figures are the ensemble-average forecasts. The open circles on the dashed line and the dotted line in the figure on the left represent the very high-resolution deterministic forecast point and the ensemble-average forecast point, respectively. The relative merit of ROC and forecast value as determined by these curves is discussed in the text.
Citation: Weather and Forecasting 23, 4; 10.1175/2007WAF2007064.1
The reservoir schematic diagram for the cost–loss economic model discussed in the text.
Citation: Weather and Forecasting 23, 4; 10.1175/2007WAF2007064.1
Profit as a function of the market–contract price ratio for different forecast configurations. The profit for each different configuration is calculated relative to a baseline maximum possible profit (if the weather forecasts were perfect).
Citation: Weather and Forecasting 23, 4; 10.1175/2007WAF2007064.1
Cost–loss contingency table of precipitation forecasts and observations. The number of hits is given by a, b refers to false alarms, c is the number of misses, and d is the number of correct rejections. Actions resulting from a forecast incur a cost C, while losses L result from each observed event that was not forecast. Correct rejections incur no cost and result in no loss. An event is when the precipitation exceeds a threshold.
Reservoir physical parameters for cost–loss calculations.
Cost–loss ratios for the reservoirs in this study.
Cost–loss contingency table summary of actual precipitation forecasts and observations for Jordan River for the 50 mm day−1 precipitation threshold. The summary includes the full ensemble, the deterministic forecasts from the MM5–4 km model, and the ensemble average. The actual sample expense, Eactual, is determined as a function of L through the cost–loss ratio for Jordan River (C/L = 0.068). The cost is estimated by evaluating L as explained in the text.
Source: U.S. Energy Information Administration (2005), Form EIA-861, Annual Electric Power Industry Report. The price quoted is the latest available average wholesale price of electricity for the United States.
Profit as used in this context is the difference between generation revenue and contract costs for undelivered energy. There are many other costs associated with hydropower generation, such as water license fees and transmission charges, that are not included in the profit calculations here.
