Verification of National Weather Service Ensemble Streamflow Predictions for Water Supply Forecasting in the Colorado River Basin

Kristie J. Franz Department of Hydrology and Water Resources, The University of Arizona, Tucson, Arizona

Search for other papers by Kristie J. Franz in
Current site
Google Scholar
PubMed
Close
,
Holly C. Hartmann Department of Hydrology and Water Resources, The University of Arizona, Tucson, Arizona

Search for other papers by Holly C. Hartmann in
Current site
Google Scholar
PubMed
Close
,
Soroosh Sorooshian Department of Hydrology and Water Resources, The University of Arizona, Tucson, Arizona

Search for other papers by Soroosh Sorooshian in
Current site
Google Scholar
PubMed
Close
, and
Roger Bales Department of Hydrology and Water Resources, The University of Arizona, Tucson, Arizona

Search for other papers by Roger Bales in
Current site
Google Scholar
PubMed
Close
Restricted access

We are aware of a technical issue preventing figures and tables from showing in some newly published articles in the full-text HTML view.
While we are resolving the problem, please use the online PDF version of these articles to view figures and tables.

Abstract

The Ensemble Streamflow Prediction (ESP) system, developed by the National Weather Service (NWS), uses conceptual hydrologic models and historical data to generate a set, or ensemble, of possible streamflow scenarios conditioned on the initial states of a given basin. Using this approach, simulated historical probabilistic forecasts were generated for 14 forecast points in the Colorado River basin, and the statistical properties of the ensembles were evaluated. The median forecast traces were analyzed using “traditional” verification measures; these forecasts represented “deterministic ESP forecasts.” The minimum-error and historical traces were examined to evaluate the median forecasts and the forecast system. Distribution-oriented verification measures were used to analyze the probabilistic information contained in the entire forecast ensemble. Using a single-trace prediction, for example, the median, resulted in a loss of valuable uncertainty information about predicted seasonal volumes that is provided by the entire ensemble. The minimum-error and historical traces revealed that there are errors in the data, calibration, and models, which are part of the uncertainty provided by the probabilistic forecasts, but are not considered in the median forecast. The simulated ESP forecasts more accurately predicted future streamflow than climatology forecasts and, on average, provided useful information about the likelihood of future streamflow magnitude with a lead time of up to 7 months. Overall, the forecast provided stronger probability statements and became more reliable at shorter lead times. The distribution-oriented verification approach was shown to be applicable to ESP outlooks and appropriate for extracting detailed performance information, although interpretation of the results is complicated by inadequate sample sizes.

Current affiliation: Department of Civil and Environmental Engineering, University of California, Irvine, Irvine, California

Current affiliation: University of California, Merced, Merced, California

Corresponding author address: Ms. Kristie J. Franz, Dept. of Civil and Environmental Engineering, University of California, Irvine, E/4130 Engineering Gateway, Irvine, CA 92697-2175. Email: franzk@uci.edu

Abstract

The Ensemble Streamflow Prediction (ESP) system, developed by the National Weather Service (NWS), uses conceptual hydrologic models and historical data to generate a set, or ensemble, of possible streamflow scenarios conditioned on the initial states of a given basin. Using this approach, simulated historical probabilistic forecasts were generated for 14 forecast points in the Colorado River basin, and the statistical properties of the ensembles were evaluated. The median forecast traces were analyzed using “traditional” verification measures; these forecasts represented “deterministic ESP forecasts.” The minimum-error and historical traces were examined to evaluate the median forecasts and the forecast system. Distribution-oriented verification measures were used to analyze the probabilistic information contained in the entire forecast ensemble. Using a single-trace prediction, for example, the median, resulted in a loss of valuable uncertainty information about predicted seasonal volumes that is provided by the entire ensemble. The minimum-error and historical traces revealed that there are errors in the data, calibration, and models, which are part of the uncertainty provided by the probabilistic forecasts, but are not considered in the median forecast. The simulated ESP forecasts more accurately predicted future streamflow than climatology forecasts and, on average, provided useful information about the likelihood of future streamflow magnitude with a lead time of up to 7 months. Overall, the forecast provided stronger probability statements and became more reliable at shorter lead times. The distribution-oriented verification approach was shown to be applicable to ESP outlooks and appropriate for extracting detailed performance information, although interpretation of the results is complicated by inadequate sample sizes.

Current affiliation: Department of Civil and Environmental Engineering, University of California, Irvine, Irvine, California

Current affiliation: University of California, Merced, Merced, California

Corresponding author address: Ms. Kristie J. Franz, Dept. of Civil and Environmental Engineering, University of California, Irvine, E/4130 Engineering Gateway, Irvine, CA 92697-2175. Email: franzk@uci.edu

Save
  • Anderson, E. A., 1973: National Weather Service River Forecast System-Snow Accumulation and Ablation Model. NOAA Tech. Memo. NWS Hydro-17, U.S. National Weather Service. [Available from Office of Hydrologic Development, NOAA/NWS, 1325 East–West Highway, Silver Spring, MD 20910.].

    • Search Google Scholar
    • Export Citation
  • Brandon, D., 1998: Forecasting streamflow in the Colorado and Great Basins using ‘El Niño’ indices as predictors in a statistical water supply forecast system. Proc. Flood Plain Management Association Spring Conference: Winter '97–'98: Year of the Great El-Nino?, San Diego, CA, Flood Plain Management Association.

    • Search Google Scholar
    • Export Citation
  • Brier, G. W., 1950: Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78 , 13.

  • Bureau of Reclamation, 1998: Colorado River System consumptive uses and losses report: 1986–1990. U.S. Department of the Interior Rep., 40 pp. [Available online at http://www.usbr.gov/uc/library/envdocs/reports/crs/reports/1986.pdf.].

    • Search Google Scholar
    • Export Citation
  • Burnash, R. J., 1995: The NWS River Forecast System—Catchment modeling. Computer Models of Watershed Hydrology, V. J. Singh, Ed., Water Resources Publications, 311–366.

    • Search Google Scholar
    • Export Citation
  • Burnash, R. J., Ferrel R. L. , and McGuire R. A. , 1973: A generalized streamflow simulation system: Conceptual modeling for digital computers. Report of the Joint Federal–State River Forecast Center, Department of Water Resources, State of California and the National Weather Service, 204 pp.

    • Search Google Scholar
    • Export Citation
  • Croley, T. E., 2000: Using Meteorology Probability Forecasts in Operational Hydrology. American Society of Civil Engineers Press, 206 pp.

    • Search Google Scholar
    • Export Citation
  • Day, G. N., 1985: Extended streamflow forecasting using NWSRFS. J. Water Resour. Plann. Manage., 111 , 157170.

  • Day, G. N., Brazil L. E. , McCarthy C. S. , and Laurine D. P. , 1992: Verification of the National Weather Service extended streamflow prediction procedure. Proc. AWRA 28th Annual Conf. and Symp., Reno, NV, American Water Resource Association, 163–172.

    • Search Google Scholar
    • Export Citation
  • Epstein, E. S., 1969: A scoring system for probability forecasts of ranked categories. J. Appl. Meteor., 8 , 985987.

  • Franz, K. J., Hartmann H. C. , Sorooshian S. , and Bales R. , 2003: Evaluation of National Weather Service ensemble streamflow predictions for the Colorado River Basin. Tech. Rep. HWR 03-010, Dept. of Hydrology and Water Resources, The University of Arizona, Tucson, AZ, 177 pp.

    • Search Google Scholar
    • Export Citation
  • Hartmann, H. C., 2001: Stakeholder driven research in a hydroclimatic context. Ph.D. dissertation, The University of Arizona, Tucson, AZ, 256 pp.

    • Search Google Scholar
    • Export Citation
  • Hartmann, H. C., Bales R. , and Sorooshian S. , 1999: Weather, climate and forecasting for the southwest U.S. Rep. Series CL2-99, Institute for the Study of Planet Earth, The University of Arizona, Tucson, AZ, 172 pp.

    • Search Google Scholar
    • Export Citation
  • Hartmann, H. C., Bales R. , and Sorooshian S. , 2002a: Weather, climate, and hydrologic forecasting for the U.S. Southwest: A survey. Climate Res., 21 , 239258.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hartmann, H. C., Pagano T. C. , Bales R. , and Sorooshian S. , 2002b: Confidence builders: Evaluating seasonal climate forecasts from user perspectives. Bull. Amer. Meteor. Soc., 83 , 683698.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., and Winkler R. L. , 1987: A general framework for forecast verification. Mon. Wea. Rev., 115 , 13301338.

  • Murphy, A. H., and Winkler R. L. , 1992: Diagnostic verification of probability forecasts. Int. J. Forecasting, 7 , 435455.

  • Murphy, A. H., Brown B. G. , and Chen Y. , 1989: Diagnostic verification of temperature forecasts. Wea. Forecasting, 4 , 485501.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Palmer, P. L., 1988: The SCS snow survey water supply forecasting program: Current operations and future directions. Proc. Western Snow Conf., Kalispell, MT, Western Snow Conference, 43–51.

    • Search Google Scholar
    • Export Citation
  • Riverside Technology, Inc., 1999: National Weather Service Extended Streamflow Prediction Verification System (ESPVS). Draft Users Manual, U.S. National Weather Service.

    • Search Google Scholar
    • Export Citation
  • Shafer, B. A., and Huddleston J. M. , 1984: Analysis of seasonal volume streamflow forecast errors in the western United States. Proc. A Critical Assessment of Forecasting in Water Quality Goals in Western Water Resources Management, Bethesda, MD, American Water Resources Association, 117–126.

    • Search Google Scholar
    • Export Citation
  • Sheppard, P. R., Comrie A. C. , Packin G. D. , Angersbach K. , and Hughes M. K. , 1999: The climate of the Southwest. CLIMAS Rep. Series CL1-99, Institute for the Study of Planet Earth, The University of Arizona, Tucson, AZ, 39 pp.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 1995: Forecast verification. Statistical Methods in the Atmospheric Sciences, Academic Press, 233–283.

  • Wilks, D. S., 2000: Diagnostic verification of the Climate Prediction Center long-lead outlooks, 1995–98. J. Climate, 13 , 23892403.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1399 514 97
PDF Downloads 651 107 14