A Nonparametric Postprocessor for Bias Correction of Hydrometeorological and Hydrologic Ensemble Forecasts

James D. Brown NOAA/National Weather Service, Office of Hydrologic Development, Silver Spring, Maryland, and University Corporation for Atmospheric Research, Boulder, Colorado

Search for other papers by James D. Brown in
Current site
Google Scholar
PubMed
Close
and
Dong-Jun Seo NOAA/National Weather Service, Office of Hydrologic Development, Silver Spring, Maryland, and University Corporation for Atmospheric Research, Boulder, Colorado

Search for other papers by Dong-Jun Seo in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

This paper describes a technique for quantifying and removing biases from ensemble forecasts of hydrometeorological and hydrologic variables. The technique makes no a priori assumptions about the distributional form of the variables, which is often unknown or difficult to model parametrically. The aim is to estimate the conditional cumulative distribution function (ccdf) of the observed variable given a (possibly biased) real-time ensemble forecast. This ccdf represents the “true” probability distribution of the forecast variable, subject to sampling uncertainties. In the absence of a known distributional form, the ccdf should be estimated nonparametrically. It is noted that the probability of exceeding a threshold of the observed variable, such as flood stage, is equivalent to the expectation of an indicator variable defined for that threshold. The ccdf is then modeled through a linear combination of the indicator variables of the forecast ensemble members. The technique is based on Bayesian optimal linear estimation of indicator variables and is analogous to indicator cokriging (ICK) in geostatistics. By developing linear estimators for the conditional expectation of the observed variable at many thresholds, ICK provides a discrete approximation of the full ccdf. Since ICK minimizes the conditional error variance of the indicator variable at each threshold, it effectively minimizes the continuous ranked probability score (CRPS) when infinitely many thresholds are employed. The technique is used to bias-correct precipitation ensemble forecasts from the NCEP Global Ensemble Forecast System (GEFS) and streamflow ensemble forecasts from the National Weather Service (NWS) River Forecast Centers (RFCs). Split-sample validation results are presented for several attributes of ensemble forecast quality, including reliability and discrimination. In general, the forecast biases were substantially reduced following ICK. Overall, the technique shows significant potential for bias-correcting ensemble forecasts whose distributional form is unknown or nonparametric.

Corresponding author address: James Brown, NOAA/NWS/OHD, 1325 East-West Highway, Silver Spring, MD 20910. Email: james.d.brown@noaa.gov

Abstract

This paper describes a technique for quantifying and removing biases from ensemble forecasts of hydrometeorological and hydrologic variables. The technique makes no a priori assumptions about the distributional form of the variables, which is often unknown or difficult to model parametrically. The aim is to estimate the conditional cumulative distribution function (ccdf) of the observed variable given a (possibly biased) real-time ensemble forecast. This ccdf represents the “true” probability distribution of the forecast variable, subject to sampling uncertainties. In the absence of a known distributional form, the ccdf should be estimated nonparametrically. It is noted that the probability of exceeding a threshold of the observed variable, such as flood stage, is equivalent to the expectation of an indicator variable defined for that threshold. The ccdf is then modeled through a linear combination of the indicator variables of the forecast ensemble members. The technique is based on Bayesian optimal linear estimation of indicator variables and is analogous to indicator cokriging (ICK) in geostatistics. By developing linear estimators for the conditional expectation of the observed variable at many thresholds, ICK provides a discrete approximation of the full ccdf. Since ICK minimizes the conditional error variance of the indicator variable at each threshold, it effectively minimizes the continuous ranked probability score (CRPS) when infinitely many thresholds are employed. The technique is used to bias-correct precipitation ensemble forecasts from the NCEP Global Ensemble Forecast System (GEFS) and streamflow ensemble forecasts from the National Weather Service (NWS) River Forecast Centers (RFCs). Split-sample validation results are presented for several attributes of ensemble forecast quality, including reliability and discrimination. In general, the forecast biases were substantially reduced following ICK. Overall, the technique shows significant potential for bias-correcting ensemble forecasts whose distributional form is unknown or nonparametric.

Corresponding author address: James Brown, NOAA/NWS/OHD, 1325 East-West Highway, Silver Spring, MD 20910. Email: james.d.brown@noaa.gov

Save
  • Ajami, N., Duan Q. , and Sorooshian S. , 2007: An integrated hydrologic Bayesian multimodel combination framework: Confronting input, parameter, and model structural uncertainty in hydrologic prediction. Water Resour. Res., 43 , W01403. doi:10.1029/2005WR004745.

    • Search Google Scholar
    • Export Citation
  • Ali, A. I., and Lall U. , 1996: A kernel estimator for stochastic subsurface characterization from drill-log data. Ground Water, 34 , 647658.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., 1996: A method for producing and evaluating probabilistic forecasts from ensemble model integrations. J. Climate, 9 , 15181530.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Anderson, M. G., and Bates P. D. , Eds. 2001: Model Validation: Perspectives in Hydrological Science. John Wiley and Sons, 512 pp.

  • Beven, K., and Binley A. , 1992: The future of distributed models: Model calibration and uncertainty prediction. Hydrol. Processes, 6 , 279298.

  • Borga, M., and Vizzaccaro A. , 1997: On the interpolation of hydrologic variables: Formal equivalence of multiquadratic surface fitting and kriging. J. Hydrol., 195 , 160171.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bradley, A. A., Schwartz S. S. , and Hashino T. , 2004: Distributions-oriented verification of ensemble streamflow predictions. J. Hydrometeor., 5 , 532545.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brier, G. W., 1950: Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78 , 13.

  • Bröcker, J., and Smith L. A. , 2007: Scoring probabilistic forecasts: On the importance of being proper. Wea. Forecasting, 22 , 382388.

  • Brown, J. D., and Heuvelink G. , 2005: Assessing uncertainty propagation through physically based models of soil water flow and solute transport. The Encyclopedia of Hydrological Sciences, M. Anderson, Ed., John Wiley and Sons, 1181–1195.

    • Search Google Scholar
    • Export Citation
  • Buja, A., Hastie T. J. , and Tibshirani R. J. , 1989: Linear smoothers and additive models. Ann. Stat., 17 , 453510.

  • Ciach, G. J., Morrissey M. L. , and Krajewski W. F. , 2000: Conditional bias in radar rainfall estimation. J. Appl. Meteor., 39 , 19411946.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cressie, N. A. C., 1993: Statistics for Spatial Data. rev. ed. John Wiley and Sons, 900 pp.

  • Demargne, J., Mullusky M. , Werner K. , Adams T. , Lindsey S. , Schwein N. , Marosi W. , and Welles E. , 2009: Application of forecast verification science to operational river forecasting in the U.S. National Weather Service. Bull. Amer. Meteor. Soc., 90 , 779784.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deutsch, C. V., and Journel A. G. , 1992: GSLIB: Geostatistical Software Library and User’s Guide. Oxford University Press, 340 pp.

  • Draper, N. R., and Smith H. , 1998: Applied Regression Analysis. 3rd ed. John Wiley and Sons, 736 pp.

  • Dubrule, O., 1983: Two methods with different objectives: Splines and kriging. Math. Geol., 15 , 245257.

  • Eckel, F. A., and Walters M. K. , 1998: Calibrated probabilistic quantitative precipitation forecasts based on the MRF ensemble. Wea. Forecasting, 13 , 11321147.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fawcett, T., 2006: An introduction to ROC analysis. Pattern Recognit. Lett., 27 , 861874.

  • Franz, K. J., Hartmann H. C. , Sorooshian S. , and Bales R. , 2003: Verification of National Weather Service ensemble streamflow predictions for water supply forecasting in the Colorado River basin. J. Hydrometeor., 4 , 11051118.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fuller, W. A., 1987: Measurement Error Models. John Wiley and Sons, 440 pp.

  • Georgakakos, K. P., 2003: Probabilistic climate-model diagnostics for hydrologic and water resources impact studies. J. Hydrometeor., 4 , 92105.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Glahn, H., and Lowry D. , 1972: The use of model output statistics (MOS) in objective weather forecasting. J. Appl. Meteor., 11 , 12031211.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gneiting, T., Raftery E. , Westveld A. H. III, and Goldman T. , 2005: Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Mon. Wea. Rev., 133 , 10981118.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gneiting, T., Balabdaoui F. , and Raftery A. E. , 2007: Probabilistic forecasts, calibration and sharpness. J. Roy. Stat. Soc., 69B , 243268.

    • Search Google Scholar
    • Export Citation
  • Golub, G. H., and Van Loan C. F. , 1996: Matrix Computations. 3rd ed. Johns Hopkins, 642 pp.

  • Goovaerts, P., 1997: Geostatistics for Natural Resource Evaluation. Oxford University Press, 483 pp.

  • Green, D. M., and Swets J. M. , 1966: Signal Detection Theory and Psychophysics. Wiley and Sons Inc., 455 pp.

  • Gupta, H. V., Beven K. J. , and Wagener T. , 2005: Model calibration and uncertainty estimation. The Encyclopedia of Hydrological Sciences, M. Anderson, Ed., John Wiley & Sons, 2015–2032.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 1997: Reliability diagrams for multicategory probabilistic forecasts. Wea. Forecasting, 12 , 736741.

  • Hamill, T. M., Whittaker J. S. , and Mullen S. L. , 2006: Reforecasts: An important dataset for improving weather predictions. Bull. Amer. Meteor. Soc., 87 , 3346.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., Hagedorn R. , and Whitaker J. S. , 2008: Probabilistic forecast calibration using ECMWF and GFS ensemble reforecasts. Part II: Precipitation. Mon. Wea. Rev., 136 , 26202632.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hansen, P. C., 1987: The truncated SVD as a method for regularization. BIT Numer. Math., 27 , 534553.

  • Hashino, T., Bradley A. A. , and Schwartz S. S. , 2006: Evaluation of bias-correction methods for ensemble streamflow volume forecasts. Hydrol. Earth Syst. Sci. Discuss., 3 , 561594.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • He, X., and Ng P. , 1999: COBS: Qualitatively constrained smoothing via linear programming. Comput. Stat., 14 , 315337.

  • Hersbach, H., 2000: Decomposition of the continuous ranked probability score for ensemble prediction systems. Wea. Forecasting, 15 , 559570.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hsu, W-R., and Murphy A. H. , 1986: The attributes diagram: A geometrical framework for assessing the quality of probability forecasts. Int. J. Forecast., 2 , 285293.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Isaaks, E. H., and Strivastava R. M. , 1989: An Introduction to Applied Geostatistics. Oxford University Press, 561 pp.

  • Jolliffe, I. T., and Stephenson D. B. , Eds. 2003: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. John Wiley and Sons, 240 pp.

    • Search Google Scholar
    • Export Citation
  • Journel, A. G., and Huijbregts Ch J. , 1978: Mining Geostatistics. Academic Press, 600 pp.

  • Katz, R. W., and Murphy A. H. , 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, 238 pp.

  • Kelly, K. S., and Krzysztofowicz R. , 1997: A bivariate meta-Gaussian density for use in hydrology. Stochastic Hydrol. Hydraul., 11 , 1731.

  • Kennedy, E. J., 1983: Computation of continuous records of streamflow. Techniques of Water-Resources Investigations of the United States Geological Survey Rep. 3-A13, 52 pp. [Available online at http://pubs.usgs.gov/twri/twri3-a13/pdf/TWRI_3-A13.pdf].

    • Search Google Scholar
    • Export Citation
  • Koenker, R., 2005: Quantile Regression. Cambridge University Press, 370 pp.

  • Larson, L. W., 1976: Precipitation and its Measurement: A State of the Art. Water Resources Series, Vol. 24, University of Wyoming, 71 pp.

    • Search Google Scholar
    • Export Citation
  • Mason, S. J., and Graham N. E. , 2002: Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation. Quart. J. Roy. Meteor. Soc., 128 , 21452166.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Matheson, J. E., and Winkler R. L. , 1976: Scoring rules for continuous probability distributions. Manage. Sci., 22 , 10871095.

  • Murphy, A. H., and Winkler R. L. , 1987: A general framework for forecast verification. Mon. Wea. Rev., 115 , 13301338.

  • NRC, 2006: Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. National Academies Press, 112 pp.

    • Search Google Scholar
    • Export Citation
  • O’Conner, P. D. T., 2002: Practical Reliability Engineering. 4th ed. John Wiley and Sons, 540 pp.

  • Olsson, J., and Lindström G. , 2008: Evaluation and calibration of operational hydrological ensemble forecasts in Sweden. J. Hydrol., 350 , 1424.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Raftery, A. E., Gneiting T. , Balabdaoui F. , and Polakowski M. , 2005: Using Bayesian model averaging to calibrate forecast ensembles. Mon. Wea. Rev., 133 , 11551174.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roulston, M. S., and Smith L. A. , 2002: Evaluating probabilistic forecasts using information theory. Mon. Wea. Rev., 130 , 16531660.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Saito, H., and Goovaerts P. , 2002: Accounting for measurement error in uncertainty modeling and decision-making using indicator kriging and p-field simulation: Application to a dioxin contaminated site. Environmetrics, 13 , 555567.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schaake, J., and Coauthors, 2007: Precipitation and temperature ensemble forecasts from single-value forecasts. Hydrol. Earth Syst. Sci., 4 , 655717.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schweppe, F. C., 1973: Uncertain Dynamic Systems. Prentice-Hall, 576 pp.

  • Seo, D-J., 1996: Nonlinear estimation of spatial distribution of rainfall - An indicator cokriging approach. Stochastic Hydrol. Hydraul., 10 , 127150.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Seo, D-J., Herr H. D. , and Schaake J. C. , 2006: A statistical post-processor for accounting of hydrologic uncertainty in short-range ensemble streamflow prediction. Hydrol. Earth Syst. Sci., 3 , 19872035.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sloughter, J. M., Raftery A. E. , Gneiting T. , and Fraley C. , 2007: Probabilistic quantitative precipitation forecasting using Bayesian model averaging. Mon. Wea. Rev., 135 , 32093220.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stensrud, D. J., Brooks H. E. , Du J. , Tracton M. S. , and Rogers E. , 1999: Using ensembles for short-range forecasting. Mon. Wea. Rev., 127 , 433446.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Talagrand, O., 1997: Assimilation of observations, an introduction. J. Meteor. Soc. Japan, 75 , 191209.

  • Toth, Z., Kalnay E. , Tracton S. M. , Wobus R. , and Irwin J. , 1997: A synoptic evaluation of the NCEP ensemble. Wea. Forecasting, 12 , 140153.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Watson, G. S., 1984: Smoothing and interpolation by kriging and with splines. Math. Geol., 16 , 601615.

  • Wei, M., Toth Z. , Wobus R. , and Zhu Y. , 2008: Initial perturbations based on the ensemble transform (ET) technique in the NCEP global operational forecast system. Tellus, 60A , 6279.

    • Search Google Scholar
    • Export Citation
  • Wilczak, J., and Coauthors, 2006: Bias-corrected ensemble and probabilistic forecasts of surface ozone over eastern North America during the summer of 2004. J. Geophys. Res., 111 , D23S28. doi:10.1029/2006JD007598.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences. Academic Press, 467 pp.

  • Wilks, D. S., 2009: Extending logistic regression to provide full-probability-distribution MOS forecasts. Meteor. Appl., 16 , 361368.

  • Yandell, B. S., 1993: Smoothing splines—A tutorial. Statistician, 42 , 317319.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 306 122 9
PDF Downloads 197 62 5