Search Results

You are looking at 1 - 9 of 9 items for

  • Author or Editor: Andrew Schepen x
  • All content x
Clear All Modify Search
Andrew Schepen and Q. J. Wang

Abstract

The majority of international climate modeling centers now produce seasonal rainfall forecasts from coupled general circulation models (GCMs). Seasonal rainfall forecasting is highly challenging, and GCM forecast accuracy is still poor for many regions and seasons. Additionally, forecast uncertainty tends to be underestimated meaning that forecast probabilities are statistically unreliable. A common strategy employed to improve the overall accuracy and reliability of GCM forecasts is to merge forecasts from multiple models into a multimodel ensemble (MME). The most widely used technique is to simply pool all of the forecast ensemble members from multiple GCMs into what is known as a superensemble. In Australia, seasonal rainfall forecasts are produced using the Predictive Ocean–Atmosphere Model for Australia (POAMA). In this paper, the authors demonstrate that mean corrected superensembles formed by merging forecasts from POAMA with those from three international models in the ENSEMBLES dataset remain poorly calibrated in many cases. The authors propose and evaluate a two-step process for producing MMEs. First, forecast calibration of the individual GCMs is carried out by using Bayesian joint probability models that account for parameter uncertainty. The calibration leads to satisfactory forecast reliability. Second, the individually calibrated forecasts of the GCMs are merged through Bayesian model averaging (BMA). The use of multiple GCMs results in better forecast accuracy, while maintaining reliability, than using POAMA only. Compared with using equal-weight averaging, BMA weighting produces sharper and more accurate forecasts.

Full access
Andrew Schepen, Q. J. Wang, and Yvette Everingham

Abstract

There are a number of challenges that must be overcome if GCM forecasts are to be widely adopted in climate-sensitive industries such as agriculture and water management. GCM outputs are frequently biased relative to observations and their ensembles are unreliable in conveying uncertainty through appropriate spread. The calibration, bridging, and merging (CBaM) method has been shown to be an effective tool for postprocessing GCM rainfall forecasts to improve ensemble forecast attributes. In this study, CBaM is modified and extended to postprocess seasonal minimum and maximum temperature forecasts from the POAMA GCM in Australia. Calibration is postprocessing GCM forecasts using a statistical model. Bridging is producing additional forecasts using statistical models that have other GCM output variables (e.g., SST) as predictors. It is demonstrated that merging calibration and bridging forecasts through CBaM effectively improves the skill of POAMA seasonal minimum and maximum temperature forecasts for Australia. It is demonstrated that CBaM produces bias-corrected forecasts that are reliable in ensemble spread and reduces forecasts to climatology when there is no evidence of forecasting skill. This work will help enable the adoption of GCM forecasts by climate-sensitive industries for quantitative modeling and decision-making.

Full access
Andrew Schepen, Q. J. Wang, and David Robertson

Abstract

Lagged oceanic and atmospheric climate indices are potentially useful predictors of seasonal rainfall totals. A rigorous Bayesian joint probability modeling approach is applied to find the cross-validation predictive densities of gridded Australian seasonal rainfall totals using lagged climate indices as predictors over the period of 1950–2009. The evidence supporting the use of each climate index as a predictor of seasonal rainfall is quantified by the pseudo-Bayes factor based on cross-validation predictive densities. The evidence strongly supports the use of climate indices from the Pacific region with weaker, but positive, evidence for the use of climate indices from the Indian region and the extratropical region. The spatial structure and seasonal variation of the evidence for each climate index is mapped and compared. Spatially, the strongest supporting evidence is found for forecasting in northern and eastern Australia. Seasonally, the strongest evidence is found from August–October to November–January and the weakest evidence is found from March–May to May–July. In some regions and seasons, there is little evidence supporting the use of climate indices for forecasting seasonal rainfall. Climate indices derived from sea surface temperature anomalies in the Pacific region show stronger persistence in the relationship with Australian seasonal rainfall totals than climate indices derived from sea surface temperature anomalies in the Indian region. Climate indices derived from atmospheric variables are also strongly supported, provided they represent the large-scale circulation. Many climate indices are found to show similar supporting evidence for forecasting Australian seasonal rainfall, leading to the prospect of combining climate indices in multiple predictor models and/or model averaging.

Full access
Andrew Schepen, Yvette Everingham, and Quan J. Wang

Abstract

Multivariate seasonal climate forecasts are increasingly required for quantitative modeling in support of natural resources management and agriculture. GCM forecasts typically require postprocessing to reduce biases and improve reliability; however, current seasonal postprocessing methods often ignore multivariate dependence. In low-dimensional settings, fully parametric methods may sufficiently model intervariable covariance. On the other hand, empirical ensemble reordering techniques can inject desired multivariate dependence in ensembles from template data after univariate postprocessing. To investigate the best approach for seasonal forecasting, this study develops and tests several strategies for calibrating seasonal GCM forecasts of rainfall, minimum temperature, and maximum temperature with intervariable dependence: 1) simultaneous calibration of multiple climate variables using the Bayesian joint probability modeling approach; 2) univariate BJP calibration coupled with an ensemble reordering method (the Schaake shuffle); and 3) transformation-based quantile mapping, which borrows intervariable dependence from the raw forecasts. Applied to Australian seasonal forecasts from the ECMWF System4 model, univariate calibration paired with empirical ensemble reordering performs best in terms of univariate and multivariate forecast verification metrics, including the energy and variogram scores. However, the performance of empirical ensemble reordering using the Schaake shuffle is influenced by the selection of historical data in constructing a dependence template. Direct multivariate calibration is the second-best method, with its far superior performance in in-sample testing vanishing in cross validation, likely because of insufficient data relative to the number of parameters. The continued development of multivariate forecast calibration methods will support the uptake of seasonal climate forecasts in complex application domains such as agriculture and hydrology.

Free access
Q. J. Wang, Andrew Schepen, and David E. Robertson

Abstract

Merging forecasts from multiple models has the potential to combine the strengths of individual models and to better represent forecast uncertainty than the use of a single model. This study develops a Bayesian model averaging (BMA) method for merging forecasts from multiple models, giving greater weights to better performing models. The study aims for a BMA method that is capable of producing relatively stable weights in the presence of significant sampling variability, leading to robust forecasts for future events. The BMA method is applied to merge forecasts from multiple statistical models for seasonal rainfall forecasts over Australia using climate indices as predictors. It is shown that the fully merged forecasts effectively combine the best skills of the models to maximize the spatial coverage of positive skill. Overall, the skill is low for the first half of the year but more positive for the second half of the year. Models in the Pacific group contribute the most skill, and models in the Indian and extratropical groups also produce useful and sometimes distinct skills. The fully merged probabilistic forecasts are found to be reliable in representing forecast uncertainty spread. The forecast skill holds well when forecast lead time is increased from 0 to 1 month. The BMA method outperforms the approach of using a model with two fixed predictors chosen a priori and the approach of selecting the best model based on predictive performance.

Full access
Andrew Schepen, Q. J. Wang, and David E. Robertson

Abstract

Coupled general circulation models (GCMs) are increasingly being used to forecast seasonal rainfall, but forecast skill is still low for many regions. GCM forecasts suffer from systematic biases, and forecast probabilities derived from ensemble members are often statistically unreliable. Hence, it is necessary to postprocess GCM forecasts to improve skill and statistical reliability. In this study, the authors compare three methods of statistically postprocessing GCM output—calibration, bridging, and a combination of calibration and bridging—as ways to treat these problems and make use of multiple GCM outputs to increase the skill of Australian seasonal rainfall forecasts. Three calibration models are established using ensemble mean rainfall from three variants of the Predictive Ocean Atmosphere Model for Australia (POAMA) version M2.4 as predictors. Six bridging models are established using POAMA forecasts of seasonal climate indices as predictors. The calibration and bridging forecasts are merged through Bayesian model averaging. Forecast attributes including skill, sharpness, and reliability are assessed through a rigorous leave-three-years-out cross-validation procedure for forecasts of 1-month lead time. While there are overlaps in skill, there are regions and seasons where the calibration or bridging forecasts are uniquely skillful. The calibration forecasts are more skillful for January–March (JFM) to June–August (JJA). The bridging forecasts are more skillful for July–September (JAS) to December–February (DJF). Merging calibration and bridging forecasts retains, and in some seasons expands, the spatial coverage of positive skill achieved by the better of the calibration forecasts and bridging forecasts individually. The statistically postprocessed forecasts show improved reliability compared to the raw forecasts.

Full access
Sarah Strazzo, Dan C. Collins, Andrew Schepen, Q. J. Wang, Emily Becker, and Liwei Jia

Abstract

Recent research demonstrates that dynamical models sometimes fail to represent observed teleconnection patterns associated with predictable modes of climate variability. As a result, model forecast skill may be reduced. We address this gap in skill through the application of a Bayesian postprocessing technique—the calibration, bridging, and merging (CBaM) method—which previously has been shown to improve probabilistic seasonal forecast skill over Australia. Calibration models developed from dynamical model reforecasts and observations are employed to statistically correct dynamical model forecasts. Bridging models use dynamical model forecasts of relevant climate modes (e.g., ENSO) as predictors of remote temperature and precipitation. Bridging and calibration models are first developed separately using Bayesian joint probability modeling and then merged using Bayesian model averaging to yield an optimal forecast. We apply CBaM to seasonal forecasts of North American 2-m temperature and precipitation from the North American Multimodel Ensemble (NMME) hindcast. Bridging is done using the model-predicted Niño-3.4 index. Overall, the fully merged CBaM forecasts achieve higher Brier skill scores and better reliability compared to raw NMME forecasts. Bridging enhances forecast skill for individual NMME member model forecasts of temperature, but does not result in significant improvements in precipitation forecast skill, possibly because the models of the NMME better represent the ENSO–precipitation teleconnection pattern compared to the ENSO–temperature pattern. These results demonstrate the potential utility of the CBaM method to improve seasonal forecast skill over North America.

Full access
Tongtiegang Zhao, James C. Bennett, Q. J. Wang, Andrew Schepen, Andrew W. Wood, David E. Robertson, and Maria-Helena Ramos

Abstract

GCMs are used by many national weather services to produce seasonal outlooks of atmospheric and oceanic conditions and fluxes. Postprocessing is often a necessary step before GCM forecasts can be applied in practice. Quantile mapping (QM) is rapidly becoming the method of choice by operational agencies to postprocess raw GCM outputs. The authors investigate whether QM is appropriate for this task. Ensemble forecast postprocessing methods should aim to 1) correct bias, 2) ensure forecasts are reliable in ensemble spread, and 3) guarantee forecasts are at least as skillful as climatology, a property called “coherence.” This study evaluates the effectiveness of QM in achieving these aims by applying it to precipitation forecasts from the POAMA model. It is shown that while QM is highly effective in correcting bias, it cannot ensure reliability in forecast ensemble spread or guarantee coherence. This is because QM ignores the correlation between raw ensemble forecasts and observations. When raw forecasts are not significantly positively correlated with observations, QM tends to produce negatively skillful forecasts. Even when there is significant positive correlation, QM cannot ensure reliability and coherence for postprocessed forecasts. Therefore, QM is not a fully satisfactory method for postprocessing forecasts where the issues of bias, reliability, and coherence pre-exist. Alternative postprocessing methods based on ensemble model output statistics (EMOS) are available that achieve not only unbiased but also reliable and coherent forecasts. This is shown with one such alternative, the Bayesian joint probability modeling approach.

Full access
Doug Richardson, Amanda S. Black, Didier P. Monselesan, Thomas S. Moore II, James S. Risbey, Andrew Schepen, Dougal T. Squire, and Carly R. Tozer

Abstract

Subseasonal forecast skill is not homogeneous in time, and prior assessment of the likely forecast skill would be valuable for end-users. We propose a method for identifying periods of high forecast confidence using atmospheric circulation patterns, with an application to southern Australia precipitation. In particular, we use archetypal analysis to derive six patterns, called archetypes, of daily 500-hPa geopotential height (Z 500) fields over Australia. We assign Z 500 reanalysis fields to the closest-matching archetype and subsequently link the archetypes to precipitation for three key regions in the Australian agriculture and energy sectors: the Murray Basin, southwest Western Australia, and western Tasmania. Using a 20-yr hindcast dataset from the European Centre for Medium-Range Weather Forecasts subseasonal-to-seasonal prediction system, we identify periods of high confidence as when hindcast Z 500 fields closely match an archetype according to a distance criterion. We compare the precipitation hindcast accuracy during these confident periods compared to normal. Considering all archetypes, we show that there is greater skill during confident periods for lead times of less than 10 days in the Murray Basin and western Tasmania, and for greater than 6 days in southwest Western Australia, although these conclusions are subject to substantial uncertainty. By breaking down the skill results for each archetype individually, we highlight how skill tends to be greater than normal for those archetypes associated with drier-than-average conditions.

Open access