Editorial Type:
Article
Article Type:
Research Article

Locally Calibrated Probabilistic Temperature Forecasting Using Geostatistical Model Averaging and Local Bayesian Model Averaging

William Kleiber Department of Statistics, University of Washington, Seattle, Washington

Search for other papers by William Kleiber in
Current site
Google Scholar
PubMed Close
,
Adrian E. Raftery Department of Statistics, University of Washington, Seattle, Washington

Search for other papers by Adrian E. Raftery in
Current site
Google Scholar
PubMed Close
,
Jeffrey Baars Department of Atmospheric Sciences, University of Washington, Seattle, Washington

Search for other papers by Jeffrey Baars in
Current site
Google Scholar
PubMed Close
,
Tilmann Gneiting Institute of Applied Mathematics, University of Heidelberg, Heidelberg, Germany

Search for other papers by Tilmann Gneiting in
Current site
Google Scholar
PubMed Close
,
Clifford F. Mass Department of Atmospheric Sciences, University of Washington, Seattle, Washington

Search for other papers by Clifford F. Mass in
Current site
Google Scholar
PubMed Close
, and
Eric Grimit 3Tier Environmental Forecast Group, Seattle, Washington

Search for other papers by Eric Grimit in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Aug 2011
DOI:
https://doi.org/10.1175/2010MWR3511.1
Page(s):
2630–2649
Restricted access

Abstract

The authors introduce two ways to produce locally calibrated grid-based probabilistic forecasts of temperature. Both start from the Global Bayesian model averaging (Global BMA) statistical postprocessing method, which has constant predictive bias and variance across the domain, and modify it to make it local. The first local method, geostatistical model averaging (GMA), computes the predictive bias and variance at observation stations and interpolates them using a geostatistical model. The second approach, Local BMA, estimates the parameters of BMA at a grid point from stations that are close to the grid point and similar to it in elevation and land use. The results of these two methods applied to the eight-member University of Washington Mesoscale Ensemble (UWME) are given for the 2006 calendar year. GMA was calibrated and sharper than Global BMA, with prediction intervals that were 8% narrower than Global BMA on average. Examples using sparse and dense training networks of stations are shown. The sparse network experiment illustrates the ability of GMA to draw information from the entire training network. The performance of Local BMA was not statistically different from Global BMA in the dense network experiment, and was superior to both GMA and Global BMA in areas with sufficient nearby training data.

Current affiliation: Institute for Mathematics Applied to Geosciences, National Center for Atmospheric Research, Boulder, Colorado.

Corresponding author address: William Kleiber, Department of Statistics, University of Washington, Box 354322, Seattle, WA 98195-4322. E-mail: wkleiber@ucar.edu

Abstract

The authors introduce two ways to produce locally calibrated grid-based probabilistic forecasts of temperature. Both start from the Global Bayesian model averaging (Global BMA) statistical postprocessing method, which has constant predictive bias and variance across the domain, and modify it to make it local. The first local method, geostatistical model averaging (GMA), computes the predictive bias and variance at observation stations and interpolates them using a geostatistical model. The second approach, Local BMA, estimates the parameters of BMA at a grid point from stations that are close to the grid point and similar to it in elevation and land use. The results of these two methods applied to the eight-member University of Washington Mesoscale Ensemble (UWME) are given for the 2006 calendar year. GMA was calibrated and sharper than Global BMA, with prediction intervals that were 8% narrower than Global BMA on average. Examples using sparse and dense training networks of stations are shown. The sparse network experiment illustrates the ability of GMA to draw information from the entire training network. The performance of Local BMA was not statistically different from Global BMA in the dense network experiment, and was superior to both GMA and Global BMA in areas with sufficient nearby training data.

Current affiliation: Institute for Mathematics Applied to Geosciences, National Center for Atmospheric Research, Boulder, Colorado.

Corresponding author address: William Kleiber, Department of Statistics, University of Washington, Box 354322, Seattle, WA 98195-4322. E-mail: wkleiber@ucar.edu
Save
Cover Monthly Weather Review
Monthly Weather Review

  • Berrocal, V. J., A. E. Raftery, and T. Gneiting, 2007: Combining spatial statistical and ensemble information in probabilistic weather forecasts. Mon. Wea. Rev., 135, 1386–1402.

    • Search Google Scholar
    • Export Citation

  • Berrocal, V. J., A. E. Gelfand, and D. M. Holland, 2009: A spatio-temporal downscaler for output from numerical models. J. Agric. Biol. Environ. Stat., 15, 176–197.

    • Search Google Scholar
    • Export Citation

  • Berrocal, V. J., A. E. Gelfand, and D. M. Holland, 2010: A bivariate-space time downscaler under space and time misalignment. Annu. Appl. Stat., 4, 1942–1975.

    • Search Google Scholar
    • Export Citation

  • Bröcker, J., and L. Smith, 2008: From ensemble forecasts to predictive distribution functions. Tellus, 60A, 663–678.

  • Buizza, R., P. L. Houtekamer, Z. Toth, G. Pellerin, M. Wei, and Y. Zhu, 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Mon. Wea. Rev., 133, 1076–1097.

    • Search Google Scholar
    • Export Citation

  • Byrd, R. H., P. Lu, J. Nocedal, and C. Zhu, 1995: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput., 16, 1190–1208.

    • Search Google Scholar
    • Export Citation

  • Chilès, J. P., and P. Delfiner, 1999: Geostatistics: Modeling Spatial Uncertainty. Wiley, 695 pp.

  • Cressie, N. A. C., 1993: Statistics for Spatial Data. rev. ed. Wiley, 900 pp.

  • Cressman, G. P., 1959: An operational objective analysis system. Mon. Wea. Rev., 87, 367–374.

  • Dee, D. P., 2005: Bias and data assimilation. Quart. J. Roy. Meteor. Soc., 131, 3323–3344.

  • Dempster, A. P., N. M. Laird, and D. B. Rubin, 1977: Maximum likelihood from incomplete data via EM algorithm. J. Roy. Stat. Soc., 39B, 1–38.

    • Search Google Scholar
    • Export Citation

  • Diebold, F. X., T. A. Gunther, and A. S. Tay, 1998: Evaluating density forecasts with applications to financial risk management. Int. Econ. Rev., 39, 863–883.

    • Search Google Scholar
    • Export Citation

  • Duan, Q., N. K. Ajami, X. Gao, and S. Sorooshian, 2007: Multi-model ensemble hydrologic prediction using Bayesian model averaging. Adv. Water Resour., 30, 1371–1386.

    • Search Google Scholar
    • Export Citation

  • Eckel, F. A., and C. F. Mass, 2005: Aspects of effective mesoscale short-range ensemble forecasting. Wea. Forecasting, 20, 328–350.

    • Search Google Scholar
    • Export Citation

  • Epstein, E. S., 1969: Stochastic dynamic prediction. Tellus, 21, 739–759.

  • Fortin, V., A. C. Favre, and M. Said, 2006: Probabilistic forecasting from ensemble prediction systems: Improving upon the best-member method by using a different weight and dressing kernel for each member. Quart. J. Roy. Meteor. Soc., 132, 1349–1370.

    • Search Google Scholar
    • Export Citation

  • Gel, Y. R., 2007: Comparative analysis of the local observation-based (LOB) method and the nonparametric regression-based method for gridded bias correction in mesoscale weather forecasting. Wea. Forecasting, 22, 1243–1256.

    • Search Google Scholar
    • Export Citation

  • Gelfand, A. E., H. J. Kim, C. F. Sirmans, and S. Banerjee, 2003: Spatial modeling with spatially varying coefficient processes. J. Amer. Stat. Assoc., 98, 387–396.

    • Search Google Scholar
    • Export Citation

  • Gelfand, A. E., S. Banerjee, and D. Gamerman, 2005: Spatial process modelling for univariate and multivariate dynamic spatial data. Environmetrics, 16, 465–479.

    • Search Google Scholar
    • Export Citation

  • Glahn, B., K. Gilbert, R. Cosgrove, D. P. Ruth, and K. Sheets, 2009a: The gridding of MOS. Wea. Forecasting, 24, 520–529.

  • Glahn, B., M. Peroutka, J. Wiedenfeld, J. Wagner, G. Zylstra, B. Schuknecht, and B. Jackson, 2009b: MOS uncertainty estimates in an ensemble framework. Mon. Wea. Rev., 137, 246–268.

    • Search Google Scholar
    • Export Citation

  • Glahn, H. R., and D. A. Lowry, 1972: The use of model output statistics (MOS) in objective weather forecasting. J. Appl. Meteor., 11, 1203–1211.

    • Search Google Scholar
    • Export Citation

  • Gneiting, T., and A. E. Raftery, 2007: Strictly proper scoring rules, prediction, and estimation. J. Amer. Stat. Assoc., 102, 359–378.

    • Search Google Scholar
    • Export Citation

  • Gneiting, T., A. E. Raftery, A. H. Westveld III, and T. Goldman, 2005: Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Mon. Wea. Rev., 133, 1098–1118.

    • Search Google Scholar
    • Export Citation

  • Gneiting, T., F. Balabdaoui, and A. E. Raftery, 2007: Probabilistic forecasts, calibration and sharpness. J. Roy. Stat. Soc., 69B, 243–268.

    • Search Google Scholar
    • Export Citation

  • Grimit, E. P., T. Gneiting, V. J. Berrocal, and N. A. Johnson, 2006: The continuous ranked probability score for circular variables and its application to mesoscale forecast ensemble verification. Quart. J. Roy. Meteor. Soc., 132, 2925–2942.

    • Search Google Scholar
    • Export Citation

  • Hacker, J. P., and D. L. Rife, 2007: A practical approach to sequential estimation of systematic error on near-surface mesoscale grids. Wea. Forecasting, 22, 1257–1273.

    • Search Google Scholar
    • Export Citation

  • Hagedorn, R., T. M. Hamill, and J. S. Whitaker, 2008: Probabilistic forecast calibration using ECMWF and GFS ensemble reforecasts. Part I: Two-meter temperatures. Mon. Wea. Rev., 136, 2608–2619.

    • Search Google Scholar
    • Export Citation

  • Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129, 550–560.

  • Hamill, T. M., and S. J. Colucci, 1997: Verification of Eta–RSM short-range ensemble forecasts. Mon. Wea. Rev., 125, 1312–1327.

    • Search Google Scholar
    • Export Citation

  • Hamill, T. M., C. Snyder, and R. E. Morss, 2000: A comparison of probabilistic forecasts from bred, singular-vector, and perturbed observation ensembles. Mon. Wea. Rev., 128, 1835–1851.

    • Search Google Scholar
    • Export Citation

  • Hamill, T. M., J. S. Whitaker, and X. Wei, 2004: Ensemble reforecasting: Improving medium-range forecast skill using retrospective forecasts. Mon. Wea. Rev., 132, 1434–1447.

    • Search Google Scholar
    • Export Citation

  • Hastie, T., and R. Tibshirani, 1993: Varying-coefficient models. J. Roy. Stat. Soc., 55B, 757–796.

  • Hersbach, H., 2000: Decomposition of the continuous ranked probability score for ensemble prediction systems. Wea. Forecasting, 15, 559–570.

    • Search Google Scholar
    • Export Citation

  • Houtekamer, P. L., and J. Derome, 1995: Methods for ensemble prediction. Mon. Wea. Rev., 123, 2181–2196.

  • Johnson, C., and R. Swinbank, 2009: Medium-range multimodel ensemble combination and calibration. Quart. J. Roy. Meteor. Soc., 135, 777–794.

    • Search Google Scholar
    • Export Citation

  • Kharin, V. V., and F. W. Zwiers, 2002: Climate predictions with multimodel ensembles. J. Climate, 15, 793–799.

  • Leith, C. E., 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev., 102, 409–418.

  • Liu, Z., N. Le, and J. V. Zidek, 2008: Combining measurements and physical model outputs for the spatial prediction of hourly ozone space–time fields. Department of Statistics, University of British Columbia, Tech. Rep. 239, 23 pp. [Available online at http://www.stat.ubc.ca/Research/TechReports/techreports/239.pdf.]

    • Search Google Scholar
    • Export Citation

  • Mass, C. F., J. Baars, G. Wedam, E. Grimit, and R. Steed, 2008: Removal of systematic model bias on a model grid. Wea. Forecasting, 23, 438–459.

    • Search Google Scholar
    • Export Citation

  • Mass, C. F., and Coauthors, 2009: PROBCAST: A Web-based portal to mesoscale probabilistic forecasts. Bull. Amer. Meteor. Soc., 90, 1009–1014.

    • Search Google Scholar
    • Export Citation

  • Matheson, J. E., and R. L. Winkler, 1976: Scoring rules for continuous probability distributions. Manage. Sci., 22, 1087–1096.

  • Molteni, R., R. Buizza, T. N. Palmer, and T. Petroliagis, 1996: The ECMWF ensemble prediction system: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122, 73–119.

    • Search Google Scholar
    • Export Citation

  • Raftery, A. E., T. Gneiting, F. Balabdaoui, and M. Polakowski, 2005: Using Bayesian model averaging to calibrate forecast ensembles. Mon. Wea. Rev., 133, 1155–1174.

    • Search Google Scholar
    • Export Citation

  • Roulston, M. S., and L. A. Smith, 2003: Combining dynamical and statistical ensembles. Tellus, 55A, 16–30.

  • Sloughter, J. M. L., A. E. Raftery, T. Gneiting, and C. Fraley, 2007: Probabilistic quantitative precipitation forecasting using Bayesian model averaging. Mon. Wea. Rev., 135, 3209–3220.

    • Search Google Scholar
    • Export Citation

  • Sloughter, J. M. L., T. Gneiting, and A. E. Raftery, 2010: Probabilistic wind speed forecasting using ensembles and Bayesian model averaging. J. Amer. Stat. Assoc., 105, 25–35.

    • Search Google Scholar
    • Export Citation

  • Smith, R. L., C. Tebaldi, D. Nychka, and L. O. Mearns, 2009: Bayesian modeling of uncertainty in ensembles of climate models. J. Amer. Stat. Assoc., 104, 97–116.

    • Search Google Scholar
    • Export Citation

  • Stein, M. L., 1999: Interpolation of Spatial Data: Some Theory for Kriging. Springer-Verlag, 247 pp.

  • Stensrud, D. J., H. E. Brooks, J. Du, M. S. Tracton, and E. Rogers, 1999: Using ensembles for short-range forecasting. Mon. Wea. Rev., 127, 433–446.

    • Search Google Scholar
    • Export Citation

  • Tebaldi, C., and R. Knutti, 2007: The use of the multi-model ensemble in probabilistic climate projections. Philos. Trans. Roy. Soc. London, 365, 2053–2075.

    • Search Google Scholar
    • Export Citation

  • Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74, 2317–2330.

    • Search Google Scholar
    • Export Citation

  • Unger, D. A., H. van den Dool, E. O’Lenic, and D. Collins, 2009: Ensemble regression. Mon. Wea. Rev., 137, 2365–2379.

  • Wang, X., and C. H. Bishop, 2005: Improvement of ensemble reliability with a new dressing kernel. Quart. J. Roy. Meteor. Soc., 131, 965–986.

    • Search Google Scholar
    • Export Citation

  • Wilks, D. S., 2009: Extending logistic regression to provide full-probability-distribution MOS forecasts. Meteor. Appl., 16, 361–368.

    • Search Google Scholar
    • Export Citation

  • Wilks, D. S., and T. M. Hamill, 2007: Comparison of ensemble-MOS methods using GFS reforecasts. Mon. Wea. Rev., 135, 2379–2390.

  • Yussouf, N., and D. J. Stensrud, 2006: Prediction of near-surface variables at independent locations from a bias-corrected ensemble forecasting system. Mon. Wea. Rev., 134, 3415–3424.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 291 139 10
PDF Downloads 197 86 7