Search Results

You are looking at 1 - 10 of 12 items for

  • Author or Editor: Tilmann Gneiting x
  • Refine by Access: All Content x
Clear All Modify Search
Chris Fraley
,
Adrian E. Raftery
, and
Tilmann Gneiting

Abstract

Bayesian model averaging (BMA) is a statistical postprocessing technique that generates calibrated and sharp predictive probability density functions (PDFs) from forecast ensembles. It represents the predictive PDF as a weighted average of PDFs centered on the bias-corrected ensemble members, where the weights reflect the relative skill of the individual members over a training period.

This work adapts the BMA approach to situations that arise frequently in practice; namely, when one or more of the member forecasts are exchangeable, and when there are missing ensemble members. Exchangeable members differ in random perturbations only, such as the members of bred ensembles, singular vector ensembles, or ensemble Kalman filter systems. Accounting for exchangeability simplifies the BMA approach, in that the BMA weights and the parameters of the component PDFs can be assumed to be equal within each exchangeable group. With these adaptations, BMA can be applied to postprocess multimodel ensembles of any composition.

In experiments with surface temperature and quantitative precipitation forecasts from the University of Washington mesoscale ensemble and ensemble Kalman filter systems over the Pacific Northwest, the proposed extensions yield good results. The BMA method is robust to exchangeability assumptions, and the BMA postprocessed combined ensemble shows better verification results than any of the individual, raw, or BMA postprocessed ensemble systems. These results suggest that statistically postprocessed multimodel ensembles can outperform individual ensemble systems, even in cases in which one of the constituent systems is superior to the others.

Full access
J. McLean Sloughter
,
Tilmann Gneiting
, and
Adrian E. Raftery

Abstract

Probabilistic forecasts of wind vectors are becoming critical as interest grows in wind as a clean and renewable source of energy, in addition to a wide range of other uses, from aviation to recreational boating. Unlike other common forecasting problems, which deal with univariate quantities, statistical approaches to wind vector forecasting must be based on bivariate distributions. The prevailing paradigm in weather forecasting is to issue deterministic forecasts based on numerical weather prediction models. Uncertainty can then be assessed through ensemble forecasts, where multiple estimates of the current state of the atmosphere are used to generate a collection of deterministic predictions. Ensemble forecasts are often uncalibrated, however, and Bayesian model averaging (BMA) is a statistical way of postprocessing these forecast ensembles to create calibrated predictive probability density functions (PDFs). It represents the predictive PDF as a weighted average of PDFs centered on the individual bias-corrected forecasts, where the weights reflect the forecasts’ relative contributions to predictive skill over a training period. In this paper the authors extend the BMA methodology to use bivariate distributions, enabling them to provide probabilistic forecasts of wind vectors. The BMA method is applied to 48-h-ahead forecasts of wind vectors over the North American Pacific Northwest in 2003 using the University of Washington mesoscale ensemble and is shown to provide better-calibrated probabilistic forecasts than the raw ensemble, which are also sharper than probabilistic forecasts derived from climatology.

Full access
Nina Schuhen
,
Thordis L. Thorarinsdottir
, and
Tilmann Gneiting

Abstract

A bivariate ensemble model output statistics (EMOS) technique for the postprocessing of ensemble forecasts of two-dimensional wind vectors is proposed, where the postprocessed probabilistic forecast takes the form of a bivariate normal probability density function. The postprocessed means and variances of the wind vector components are linearly bias-corrected versions of the ensemble means and ensemble variances, respectively, and the conditional correlation between the wind components is represented by a trigonometric function of the ensemble mean wind direction. In a case study on 48-h forecasts of wind vectors over the North American Pacific Northwest with the University of Washington Mesoscale Ensemble, the bivariate EMOS density forecasts were calibrated and sharp, and showed considerable improvement over the raw ensemble and reference forecasts, including ensemble copula coupling.

Full access
Veronica J. Berrocal
,
Adrian E. Raftery
, and
Tilmann Gneiting

Abstract

Forecast ensembles typically show a spread–skill relationship, but they are also often underdispersive, and therefore uncalibrated. Bayesian model averaging (BMA) is a statistical postprocessing method for forecast ensembles that generates calibrated probabilistic forecast products for weather quantities at individual sites. This paper introduces the spatial BMA technique, which combines BMA and the geostatistical output perturbation (GOP) method, and extends BMA to generate calibrated probabilistic forecasts of whole weather fields simultaneously, rather than just weather events at individual locations. At any site individually, spatial BMA reduces to the original BMA technique. The spatial BMA method provides statistical ensembles of weather field forecasts that take the spatial structure of observed fields into account and honor the flow-dependent information contained in the dynamical ensemble. The members of the spatial BMA ensemble are obtained by dressing the weather field forecasts from the dynamical ensemble with simulated spatially correlated error fields, in proportions that correspond to the BMA weights for the member models in the dynamical ensemble. Statistical ensembles of any size can be generated at minimal computational cost. The spatial BMA technique was applied to 48-h forecasts of surface temperature over the Pacific Northwest in 2004, using the University of Washington mesoscale ensemble. The spatial BMA ensemble generally outperformed the BMA and GOP ensembles and showed much better verification results than the raw ensemble, both at individual sites, for weather field forecasts, and for forecasts of composite quantities, such as average temperature in National Weather Service forecast zones and minimum temperature along the Interstate 90 Mountains to Sound Greenway.

Full access
Tilmann Gneiting
,
Adrian E. Raftery
,
Anton H. Westveld III
, and
Tom Goldman

Abstract

Ensemble prediction systems typically show positive spread-error correlation, but they are subject to forecast bias and dispersion errors, and are therefore uncalibrated. This work proposes the use of ensemble model output statistics (EMOS), an easy-to-implement postprocessing technique that addresses both forecast bias and underdispersion and takes into account the spread-skill relationship. The technique is based on multiple linear regression and is akin to the superensemble approach that has traditionally been used for deterministic-style forecasts. The EMOS technique yields probabilistic forecasts that take the form of Gaussian predictive probability density functions (PDFs) for continuous weather variables and can be applied to gridded model output. The EMOS predictive mean is a bias-corrected weighted average of the ensemble member forecasts, with coefficients that can be interpreted in terms of the relative contributions of the member models to the ensemble, and provides a highly competitive deterministic-style forecast. The EMOS predictive variance is a linear function of the ensemble variance. For fitting the EMOS coefficients, the method of minimum continuous ranked probability score (CRPS) estimation is introduced. This technique finds the coefficient values that optimize the CRPS for the training data. The EMOS technique was applied to 48-h forecasts of sea level pressure and surface temperature over the North American Pacific Northwest in spring 2000, using the University of Washington mesoscale ensemble. When compared to the bias-corrected ensemble, deterministic-style EMOS forecasts of sea level pressure had root-mean-square error 9% less and mean absolute error 7% less. The EMOS predictive PDFs were sharp, and much better calibrated than the raw ensemble or the bias-corrected ensemble.

Full access
Adrian E. Raftery
,
Tilmann Gneiting
,
Fadoua Balabdaoui
, and
Michael Polakowski

Abstract

Ensembles used for probabilistic weather forecasting often exhibit a spread-error correlation, but they tend to be underdispersive. This paper proposes a statistical method for postprocessing ensembles based on Bayesian model averaging (BMA), which is a standard method for combining predictive distributions from different sources. The BMA predictive probability density function (PDF) of any quantity of interest is a weighted average of PDFs centered on the individual bias-corrected forecasts, where the weights are equal to posterior probabilities of the models generating the forecasts and reflect the models' relative contributions to predictive skill over the training period. The BMA weights can be used to assess the usefulness of ensemble members, and this can be used as a basis for selecting ensemble members; this can be useful given the cost of running large ensembles. The BMA PDF can be represented as an unweighted ensemble of any desired size, by simulating from the BMA predictive distribution.

The BMA predictive variance can be decomposed into two components, one corresponding to the between-forecast variability, and the second to the within-forecast variability. Predictive PDFs or intervals based solely on the ensemble spread incorporate the first component but not the second. Thus BMA provides a theoretical explanation of the tendency of ensembles to exhibit a spread-error correlation but yet be underdispersive.

The method was applied to 48-h forecasts of surface temperature in the Pacific Northwest in January–June 2000 using the University of Washington fifth-generation Pennsylvania State University–NCAR Mesoscale Model (MM5) ensemble. The predictive PDFs were much better calibrated than the raw ensemble, and the BMA forecasts were sharp in that 90% BMA prediction intervals were 66% shorter on average than those produced by sample climatology. As a by-product, BMA yields a deterministic point forecast, and this had root-mean-square errors 7% lower than the best of the ensemble members and 8% lower than the ensemble mean. Similar results were obtained for forecasts of sea level pressure. Simulation experiments show that BMA performs reasonably well when the underlying ensemble is calibrated, or even overdispersed.

Full access
Peter Vogel
,
Peter Knippertz
,
Andreas H. Fink
,
Andreas Schlueter
, and
Tilmann Gneiting

Abstract

Accumulated precipitation forecasts are of high socioeconomic importance for agriculturally dominated societies in northern tropical Africa. In this study, the performance of nine operational global ensemble prediction systems (EPSs) is analyzed relative to climatology-based forecasts for 1–5-day accumulated precipitation based on the monsoon seasons during 2007–14 for three regions within northern tropical Africa. To assess the full potential of raw ensemble forecasts across spatial scales, state-of-the-art statistical postprocessing methods were applied in the form of Bayesian model averaging (BMA) and ensemble model output statistics (EMOS), and results were verified against station and spatially aggregated, satellite-based gridded observations. Raw ensemble forecasts are uncalibrated and unreliable, and often underperform relative to climatology, independently of region, accumulation time, monsoon season, and ensemble. The differences between raw ensemble and climatological forecasts are large and partly stem from poor prediction for low precipitation amounts. BMA and EMOS postprocessed forecasts are calibrated, reliable, and strongly improve on the raw ensembles but, somewhat disappointingly, typically do not outperform climatology. Most EPSs exhibit slight improvements over the period 2007–14, but overall they have little added value compared to climatology. The suspicion is that parameterization of convection is a potential cause for the sobering lack of ensemble forecast skill in a region dominated by mesoscale convective systems.

Open access
Peter Vogel
,
Peter Knippertz
,
Andreas H. Fink
,
Andreas Schlueter
, and
Tilmann Gneiting

Abstract

Precipitation forecasts are of large societal value in the tropics. Here, we compare 1–5-day ensemble predictions from the European Centre for Medium-Range Weather Forecasts (ECMWF, 2009–17) and the Meteorological Service of Canada (MSC, 2009–16) over 30°S–30°N with an extended probabilistic climatology based on the Tropical Rainfall Measuring Mission 3 B42 gridded dataset. Both models predict rainfall occurrence better than the reference only over about half of all land points, with a better performance by MSC. After applying the postprocessing technique ensemble model output statistics, this fraction increases to 87% (ECMWF) and 82% (MSC). For rainfall amount there is skill in many tropical areas (about 60% of land points), which can be increased by postprocessing to 97% (ECMWF) and 88% (MSC). Forecasts for extremes (>20 mm) are only marginally worse than those of occurrence but do not improve as much through postprocessing, particularly over dry areas. Forecast performance is generally best over arid Australia and worst over oceanic deserts, the Andes and Himalayas, as well as over tropical Africa, where models misrepresent the high degree of convective organization, such that even postprocessed forecasts are hardly better than climatology. Skill of 5-day accumulated forecasts often exceeds that of shorter ranges, as timing errors matter less. An increase in resolution and major model update in 2010 has significantly improved ECMWF predictions. Especially over tropical Africa new techniques such as convection-permitting models or combined statistical-dynamical forecasts may be needed to generate skill beyond the climatological reference.

Open access
William Kleiber
,
Adrian E. Raftery
,
Jeffrey Baars
,
Tilmann Gneiting
,
Clifford F. Mass
, and
Eric Grimit

Abstract

The authors introduce two ways to produce locally calibrated grid-based probabilistic forecasts of temperature. Both start from the Global Bayesian model averaging (Global BMA) statistical postprocessing method, which has constant predictive bias and variance across the domain, and modify it to make it local. The first local method, geostatistical model averaging (GMA), computes the predictive bias and variance at observation stations and interpolates them using a geostatistical model. The second approach, Local BMA, estimates the parameters of BMA at a grid point from stations that are close to the grid point and similar to it in elevation and land use. The results of these two methods applied to the eight-member University of Washington Mesoscale Ensemble (UWME) are given for the 2006 calendar year. GMA was calibrated and sharper than Global BMA, with prediction intervals that were 8% narrower than Global BMA on average. Examples using sparse and dense training networks of stations are shown. The sparse network experiment illustrates the ability of GMA to draw information from the entire training network. The performance of Local BMA was not statistically different from Global BMA in the dense network experiment, and was superior to both GMA and Global BMA in areas with sufficient nearby training data.

Full access
J. Mc Lean Sloughter
,
Adrian E. Raftery
,
Tilmann Gneiting
, and
Chris Fraley

Abstract

Bayesian model averaging (BMA) is a statistical way of postprocessing forecast ensembles to create predictive probability density functions (PDFs) for weather quantities. It represents the predictive PDF as a weighted average of PDFs centered on the individual bias-corrected forecasts, where the weights are posterior probabilities of the models generating the forecasts and reflect the forecasts’ relative contributions to predictive skill over a training period. It was developed initially for quantities whose PDFs can be approximated by normal distributions, such as temperature and sea level pressure. BMA does not apply in its original form to precipitation, because the predictive PDF of precipitation is nonnormal in two major ways: it has a positive probability of being equal to zero, and it is skewed. In this study BMA is extended to probabilistic quantitative precipitation forecasting. The predictive PDF corresponding to one ensemble member is a mixture of a discrete component at zero and a gamma distribution. Unlike methods that predict the probability of exceeding a threshold, BMA gives a full probability distribution for future precipitation. The method was applied to daily 48-h forecasts of 24-h accumulated precipitation in the North American Pacific Northwest in 2003–04 using the University of Washington mesoscale ensemble. It yielded predictive distributions that were calibrated and sharp. It also gave probability of precipitation forecasts that were much better calibrated than those based on consensus voting of the ensemble members. It gave better estimates of the probability of high-precipitation events than logistic regression on the cube root of the ensemble mean.

Full access