Evaluation and Benchmarking of Operational Short-Range Ensemble Mean and Median Streamflow Forecasts for the Ohio River Basin

Thomas E. Adams III Department of Civil and Environmental Engineering, Virginia Polytechnic Institute and State University, Blacksburg, Virginia, and NOAA/NWS Ohio River Forecast Center, Wilmington, Ohio

Search for other papers by Thomas E. Adams III in
Current site
Google Scholar
PubMed
Close
and
Randel Dymond Department of Civil and Environmental Engineering, Virginia Polytechnic Institute and State University, Blacksburg, Virginia

Search for other papers by Randel Dymond in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

This study presents findings from a real-time forecast experiment that compares legacy deterministic hydrologic stage forecasts to ensemble mean and median stage forecasts from the NOAA/NWS Meteorological Model-Based Ensemble Forecast System (MMEFS). The NOAA/NWS Ohio River Forecast Center (OHRFC) area of responsibility defines the experimental region. Real-time forecasts from subbasins at 54 forecast point locations, ranging in drainage area, geographic location within the Ohio River valley, and watershed response time serve as the basis for analyses. In the experiment, operational hydrologic forecasts, with a 24-h quantitative precipitation forecast (QPF) and forecast temperatures, are compared to MMEFS hydrologic ensemble mean and median forecasts, with model forcings from the NOAA/NWS National Centers for Environmental Prediction (NCEP) North American Ensemble Forecast System (NAEFS), over the period from 30 November 2010 through 24 May 2012. Experiments indicate that MMEFS ensemble mean and median forecasts exhibit lower errors beginning at about lead time 90 h when forecasts at all locations are aggregated. With fast response basins that peak at ≤24 h, ensemble mean and median forecasts exhibit lower errors much earlier, beginning at about lead time 36 h, which suggests the viability of using MMEFS ensemble forecasts as an alternative to OHRFC legacy forecasts. Analyses show that ensemble median forecasts generally exhibit smaller errors than ensemble mean forecasts for all stage ranges. Verification results suggest that OHRFC MMEFS NAEFS ensemble forecasts are reasonable, but needed improvements are identified.

© 2018 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Thomas E. Adams III, tea@terrapredictions.org

Abstract

This study presents findings from a real-time forecast experiment that compares legacy deterministic hydrologic stage forecasts to ensemble mean and median stage forecasts from the NOAA/NWS Meteorological Model-Based Ensemble Forecast System (MMEFS). The NOAA/NWS Ohio River Forecast Center (OHRFC) area of responsibility defines the experimental region. Real-time forecasts from subbasins at 54 forecast point locations, ranging in drainage area, geographic location within the Ohio River valley, and watershed response time serve as the basis for analyses. In the experiment, operational hydrologic forecasts, with a 24-h quantitative precipitation forecast (QPF) and forecast temperatures, are compared to MMEFS hydrologic ensemble mean and median forecasts, with model forcings from the NOAA/NWS National Centers for Environmental Prediction (NCEP) North American Ensemble Forecast System (NAEFS), over the period from 30 November 2010 through 24 May 2012. Experiments indicate that MMEFS ensemble mean and median forecasts exhibit lower errors beginning at about lead time 90 h when forecasts at all locations are aggregated. With fast response basins that peak at ≤24 h, ensemble mean and median forecasts exhibit lower errors much earlier, beginning at about lead time 36 h, which suggests the viability of using MMEFS ensemble forecasts as an alternative to OHRFC legacy forecasts. Analyses show that ensemble median forecasts generally exhibit smaller errors than ensemble mean forecasts for all stage ranges. Verification results suggest that OHRFC MMEFS NAEFS ensemble forecasts are reasonable, but needed improvements are identified.

© 2018 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Thomas E. Adams III, tea@terrapredictions.org
Save
  • Adams, T. E., 2016: Flood forecasting in the United States NOAA/National Weather Service. Flood Forecasting: A Global Perspective, T. E. Adams and T. C. Pagano, Eds., 1st ed. Elsevier, 249–310.

    • Crossref
    • Export Citation
  • Adams, T. E., and G. Smith, 1993: National weather service interactive river forecasting using state, parameter, and data modifications. Proc. Int. Symp. Engineering Hydrology, San Francisco, CA, Environmental Water Resources Institute, 10 pp.

  • Adams, T. E., and J. Ostrowski, 2010: Short lead-time hydrologic ensemble forecasts from numerical weather prediction model ensembles. Proc. World Environmental and Water Resources Congress 2010, Providence, RI, Environmental Water Resources Institute, 2294–2304, https://doi.org/10.1061/41114(371)237.

    • Crossref
    • Export Citation
  • Adams, T. E., and T. C. Pagano, Eds., 2016: Flood Forecasting: A Global Perspective. 1st ed., Elsevier, 478 pp.

    • Crossref
    • Export Citation
  • Adams, T. E., S. Chen, and R. Dymond, 2018: Results from operational hydrologic forecasts using the NOAA/NWS OHRFC Ohio River Community HEC-RAS Model. J. Hydrol. Eng., 23, 04018028, https://doi.org/10.1061/(ASCE)HE.1943-5584.0001663.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Anderson, E. A., 1973: National Weather Service River Forecast System—Snow accumulation and ablation model. NOAA Tech. Rep. NWS-HYDRO-17, 229 pp.

  • Anderson, E. A., 2002: Calibration of conceptual hydrologic models for use in river forecasting. NWS Tech. Rep., 419 pp., http://www.nws.noaa.gov/oh/hrl/modelcalibration/1.%20Calibration%20Process/1_Anderson_CalbManual.pdf.

  • Bogner, K., and M. Kalas, 2008: Error-correction methods and evaluation of an ensemble based hydrological forecasting system for the Upper Danube catchment. Atmos. Sci. Lett., 9, 95102, https://doi.org/10.1002/asl.180.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bradley, A. A., and S. S. Schwartz, 2011: Summary verification measures and their interpretation for ensemble forecasts. Mon. Wea. Rev., 139, 30753089, https://doi.org/10.1175/2010MWR3305.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brown, J. D., and D.-J. Seo, 2010: A nonparametric postprocessor for bias correction of hydrometeorological and hydrologic ensemble forecasts. J. Hydrometeor., 11, 642665, https://doi.org/10.1175/2009JHM1188.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brown, J. D., J. Demargne, D.-J. Seo, and Y. Liu, 2010: The Ensemble Verification System (EVS): A software tool for verifying ensemble forecasts of hydrometeorological and hydrologic variables at discrete locations. Environ. Modell. Software, 25, 854872, https://doi.org/10.1016/j.envsoft.2010.01.009.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Burnash, R., 1995: The NWS River Forecast System—Catchment modeling. Computer Models of Watershed Hydrology, 1st ed., V. Singh, Ed., Water Resources Publication, 311–366.

  • Burnash, R., R. Ferral, and R. McGuire, 1973: A generalized streamflow simulation system: Conceptual modeling for digital computers. NWS Tech. Rep., 204 pp.

  • Candille, G., 2009: The multiensemble approach: The NAEFS example. Mon. Wea. Rev., 137, 16551665, https://doi.org/10.1175/2008MWR2682.1.

  • Cloke, H., and F. Pappenberger, 2009: Ensemble flood forecasting: A review. J. Hydrol., 375, 613626, https://doi.org/10.1016/j.jhydrol.2009.06.005.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cuo, L., T. C. Pagano, and Q. J. Wang, 2011: A review of quantitative precipitation forecasts and their use in short- to medium-range streamflow forecasting. J. Hydrometeor., 12, 713728, https://doi.org/10.1175/2011JHM1347.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Day, G. N., 1985: Extended streamflow forecasting using NWSRFS. J. Water Resour. Plann. Manage., 111, 157170, https://doi.org/10.1061/(ASCE)0733-9496(1985)111:2(157).

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deltares, 2018: Flood Early Warning System (FEWS). Deltares, https://www.deltares.nl/en/software/flood-forecasting-system-delft-fews-2.

  • Demargne, J., M. Mulluski, K. Werner, T. Adams, S. Lindsey, N. Schwein, W. Marosi, and E. Welles, 2009: Application of forecast verification science to operational river forecasting in the U.S. National Weather Service. Bull. Amer. Meteor. Soc., 90, 779784, https://doi.org/10.1175/2008BAMS2619.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Demargne, J., J. Brown, D.-J. Seo, L. Wu, Z. Toth, and Y. Zhu, 2010: Diagnostic verification of hydrometeorological and hydrologic ensembles. Atmos. Sci. Lett., 11, 114122, https://doi.org/10.1002/asl.261.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Demargne, J., and Coauthors, 2014: The science of NOAA’s operational hydrologic ensemble forecast service. Bull. Amer. Meteor. Soc., 95, 7998, https://doi.org/10.1175/BAMS-D-12-00081.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Demeritt, D., S. Nobert, H. Cloke, and F. Pappenberger, 2010: Challenges in communicating and using ensembles in operational flood forecasting. Meteor. Appl., 17, 209222, https://doi.org/10.1002/met.194.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Diomede, T., C. Marsigli, A. Montani, F. Nerozzi, and T. Paccagnella, 2014: Calibration of limited-area ensemble precipitation forecasts for hydrological predictions. Mon. Wea. Rev., 142, 21762197, https://doi.org/10.1175/MWR-D-13-00071.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Du, J., G. DiMego, M. S. Tracton, and B. Zhou, 2003: NCEP short-range ensemble forecasting (SREF) system: Multi-IC, multi-model and multi-physics approach. WMO/TD 1161, 5.09–5.10.

  • Georgakakos, K. P., and M. D. Hudlow, 1984: Quantitative precipitation forecast techniques for use in hydrologic forecasting. Bull. Amer. Meteor. Soc., 65, 11861200, https://doi.org/10.1175/1520-0477(1984)065<1186:QPFTFU>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129, 550560, https://doi.org/10.1175/1520-0493(2001)129<0550:IORHFV>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hartmann, H. C., T. C. Pagano, S. Sorooshian, and R. Bales, 2002: Confidence builders: Evaluating seasonal climate forecasts from user perspectives. Bull. Amer. Meteor. Soc., 83, 683698, https://doi.org/10.1175/1520-0477(2002)083<0683:CBESCF>2.3.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hashino, T., A. A. Bradley, and S. S. Schwartz, 2007: Evaluation of bias-correction methods for ensemble streamflow volume forecasts. Hydrol. Earth Syst. Sci., 11, 939950, https://doi.org/10.5194/hess-11-939-2007.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • He, Y., Y. Zhang, R. Kuligowski, R. Cifelli, and D. Kitzmiller, 2018: Incorporating satellite precipitation estimates into a radar-gauge multi-sensor precipitation estimation algorithm. Remote Sens., 10, 117, https://doi.org/10.3390/rs10010117.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Johnson, C., and N. Bowler, 2009: On the reliability and calibration of ensemble forecasts. Mon. Wea. Rev., 137, 17171720, https://doi.org/10.1175/2009MWR2715.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joslyn, S., and S. Savelli, 2010: Communicating forecast uncertainty: Public perception of weather forecast uncertainty. Meteor. Appl., 17, 180195, https://doi.org/10.1002/met.190.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kharin, V. V., and F. W. Zwiers, 2003: On the ROC score of probability forecasts. J. Climate, 16, 41454150, https://doi.org/10.1175/1520-0442(2003)016<4145:OTRSOP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kitzmiller, D., D. Miller, R. Fulton, and F. Ding, 2013: Radar and multisensor precipitation estimation techniques in National Weather Service hydrologic operations. J. Hydrol. Eng., 18, 133142, https://doi.org/10.1061/(ASCE)HE.1943-5584.0000523.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Krzysztofowicz, R., 1998: Probabilistic hydrometeorological forecasts: Toward a new era in operational forecasting. Bull. Amer. Meteor. Soc., 79, 243252, https://doi.org/10.1175/1520-0477(1998)079<0243:PHFTAN>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Li, J., Y. Chen, H. Wang, J. Qin, J. Li, and S. Chiao, 2017: Extending flood forecasting lead time in a large watershed by coupling WRF QPF with a distributed hydrological model. Hydrol. Earth Syst. Sci., 21, 12791294, https://doi.org/10.5194/hess-21-1279-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Li, M., Q. J. Wang, J. C. Bennett, and D. E. Robertson, 2016: Error reduction and representation in stages (ERRIS) in hydrological modelling for ensemble streamflow forecasting. Hydrol. Earth Syst. Sci., 20, 35613579, https://doi.org/10.5194/hess-20-3561-2016.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Michaels, S., 2015: Probabilistic forecasting and the reshaping of food risk management. J. Nat. Resour. Policy Res., 7, 4151, https://doi.org/10.1080/19390459.2014.970800.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morss, R. E., J. L. Demuth, and J. K. Lazo, 2008: Communicating uncertainty in weather forecasts: A survey of the U.S. public. Wea. Forecasting, 23, 974991, https://doi.org/10.1175/2008WAF2007088.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1991: Probabilities, odds, and forecasts of rare events. Wea. Forecasting, 6, 302307, https://doi.org/10.1175/1520-0434(1991)006<0302:POAFOR>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mylne, K. R., R. E. Evans, and R. T. Clark, 2002: Multi-model multi-analysis ensembles in quasi-operational medium-range forecasting. Quart. J. Roy. Meteor. Soc., 128, 361384, https://doi.org/10.1256/00359000260498923.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • National Academies of Sciences, Engineering, and Medicine, 2018: Integrating Social and Behavioral Sciences within the Weather Enterprise. The National Academies Press, 198 pp., https://doi.org/10.17226/24865.

    • Crossref
    • Export Citation
  • National Research Council, 1997: An Assessment of the Advanced Weather Interactive Processing System: Operational Test and Evaluation of the First system build. The National Academies Press, 124 pp. https://doi.org/10.17226/5995.

    • Crossref
    • Export Citation
  • National Research Council, 2006a: Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. The National Academies Press, 124 pp., https://doi.org/10.17226/11699.

    • Crossref
    • Export Citation
  • National Research Council, 2006b: Toward A New Advanced Hydrologic Prediction Service (AHPS). The National Academies Press, 68 pp., https://doi.org/10.17226/11598.

    • Crossref
    • Export Citation
  • NCAR, 2015: Verification: Weather forecast verification utilities, r package version 1.42. NCAR Research Applications Laboratory, https://CRAN.R-project.org/package=verification.

  • Novak, D. R., C. Bailey, K. F. Brill, P. Burke, W. A. Hogsett, R. Rausch, and M. Schichtel, 2014: Precipitation and temperature forecast performance at the Weather Prediction Center. Wea. Forecasting, 29, 489504, https://doi.org/10.1175/WAF-D-13-00066.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Office of Hydrologic Development, 2000: National Weather Service Verification software users’ manual. NOAA Tech. Rep., 26 pp., http://www.nws.noaa.gov/oh/hrl/verification/ob3/VerifyUsersManual_ob3.pdf.

  • Pappenberger, F., E. Stephens, J. Thielen, P. Salamon, D. Demeritt, S. J. van Andel, F. Wetterhall, and L. Alfieri, 2013: Visualizing probabilistic flood forecast information: Expert preferences and perceptions of best practice in uncertainty communication. Hydrol. Processes, 27, 132146, https://doi.org/10.1002/hyp.9253.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ramos, M. H., S. J. van Andel, and F. Pappenberger, 2013: Do probabilistic forecasts lead to better decisions? Hydrol. Earth Syst. Sci., 17, 22192232, https://doi.org/10.5194/hess-17-2219-2013.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rayner, S., D. Lach, and H. Ingram, 2005: Weather forecasts are for wimps: Why water resource managers do not use climate forecasts. Climatic Change, 69, 197227, https://doi.org/10.1007/s10584-005-3148-z.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • R Core Team, 2017: R: A language and environment for statistical computing. R Foundation for Statistical Computing, https://www.R-project.org.

  • Smith, M. B., D. P. Laurine, V. I. Koren, S. M. Reed, and Z. Zhang, 2013: Hydrologic model calibration in the National Weather Service. Calibration of Watershed Models, Q. Duan et al., Eds., Vol. 6, Water Science and Application Series, Amer. Geophys. Union, 133–152, https://doi.org/10.1029/WS006p0133.

    • Crossref
    • Export Citation
  • Sokol, Z., 2003: MOS-based precipitation forecasts for river basins. Wea. Forecasting, 18, 769781, https://doi.org/10.1175/1520-0434(2003)018<0769:MPFFRB>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stern, P. C., and W. E. Easterling, Eds., 1999: Making Climate Forecasts Matter. The National Academies Press, 192 pp., https://doi.org/10.17226/6370.

    • Crossref
    • Export Citation
  • U.S. Department of Commerce, 1972: National Weather Service river forecast procedures. NOAA Tech. Rep., NWS-Hydro-14, 252 pp.

  • Welles, E., S. Sorooshian, G. Carter, and B. Olsen, 2007: Hydrologic verification: A call for action and collaboration. Bull. Amer. Meteor. Soc., 88, 503511, https://doi.org/10.1175/BAMS-88-4-503.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wentao, L., D. Qingyun, M. Chiyuan, Y. Aizhong, G. Wei, and D. Zhenhua, 2017: A review on statistical postprocessing methods for hydrometeorological ensemble forecasting. Wiley Interdiscip. Rev.: Water, 4, e1246, https://doi.org/10.1002/wat2.1246.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. 2nd ed. International Geophysics Series, Vol. 100, Academic Press, 648 pp.

  • Yuan, X., and E. F. Wood, 2012: Downscaling precipitation or bias-correcting streamflow? Some implications for coupled general circulation model (CGCM)-based ensemble seasonal hydrologic forecast. Water Resour. Res., 48, W12519, https://doi.org/10.1029/2012WR012256.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhang, Y., S. Reed, and D. Kitzmiller, 2011: Effects of retrospective gauge-based readjustment of multisensor precipitation estimates on hydrologic simulations. J. Hydrometeor., 12, 429443, https://doi.org/10.1175/2010JHM1200.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 342 72 8
PDF Downloads 246 56 9