• Barnett, T. P., , Latif M. , , Graham N. E. , , Flugel M. , , Pazan S. , , and White W. B. , 1993: ENSO and ENSO-related predictability. Part I: Prediction of equatorial Pacific sea surface temperature with a hybrid coupled ocean–atmosphere model. J. Climate, 6, 15451566.

    • Search Google Scholar
    • Export Citation
  • Barnston, A. G., , Thiao W. , , and Kumar V. , 1996: Long-lead forecasts of seasonal precipitation in Africa using CCA. Wea. Forecasting, 11, 506520.

    • Search Google Scholar
    • Export Citation
  • Brown, B. H., , and Murphy A. H. , 1996: Improving forecasting performance by combining forecasts: The example of road-surface temperature forecasts. Meteor. Appl., 3, 257265.

    • Search Google Scholar
    • Export Citation
  • Copsey, D., , Sutton R. , , and Knight J. R. , 2006: Recent trends in sea level pressure in the Indian Ocean region. Geophys. Res. Lett., 33, L19712, doi:10.1029/2006GL027175.

    • Search Google Scholar
    • Export Citation
  • DeWitt, D. G., 2005: Retrospective forecasts of interannual sea surface temperature anomalies from 1982 to present using a directly coupled atmosphere–ocean general circulation model. Mon. Wea. Rev., 133, 29722995.

    • Search Google Scholar
    • Export Citation
  • Engelbrecht, F. A., , McGregor J. L. , , and Rautenbach C. J.deW. , 2007: On the development of a new nonhydrostatic atmospheric model in South Africa. S. Afr. J. Sci., 103, 127134.

    • Search Google Scholar
    • Export Citation
  • Gates, W. L., 1992: AMIP: The Atmospheric Model Intercomparison Project. Bull. Amer. Meteor. Soc., 73, 19621970.

  • Goddard, L., , and Graham N. E. , 1999: Importance of the Indian Ocean for simulating rainfall anomalies over eastern and southern Africa. J. Geophys. Res., 104 (D16), 19 09919 116.

    • Search Google Scholar
    • Export Citation
  • Goddard, L., , and Mason S. J. , 2002: Sensitivity of seasonal climate forecasts to persisted SST anomalies. Climate Dyn., 19, 619631.

  • Graham, R. J., , Evans A. D. L. , , Milne K. R. , , Harrison M. S. J. , , and Robertson K. B. , 2000: As assessment of seasonal predictability using atmospheric general circulation models. Quart. J. Roy. Meteor. Soc., 126, 22112240.

    • Search Google Scholar
    • Export Citation
  • Graham, R. J., , Gordon M. , , McLean P. J. , , Ineson S. , , Huddleston M. R. , , Davey M. K. , , Brookshaw A. , , and Barnes R. T. H. , 2005: A performance comparison of coupled and uncoupled versions of the Met Office seasonal prediction general circulation model. Tellus, 57A, 320319.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 1997: Reliability diagrams for multicategory probabilistic forecasts. Wea. Forecasting, 12, 736741.

  • Hurrel, J., , Meehl G. A. , , Bader D. , , Delworth T. L. , , Kirtman B. , , and Wielicki B. , 2009: A unified modeling approach to climate system prediction. Bull. Amer. Meteor. Soc., 90, 18191832.

    • Search Google Scholar
    • Export Citation
  • Jha, B., , and Kumar A. , 2009: A comparison of the atmospheric response to ENSO in coupled and uncoupled model simulations. Mon. Wea. Rev., 137, 479487.

    • Search Google Scholar
    • Export Citation
  • Jury, M. R., 1996: Regional teleconnection patterns associated with summer rainfall over South Africa, Namibia and Botswana. Int. J. Climatol., 16, 135153.

    • Search Google Scholar
    • Export Citation
  • Jury, M. R., , Mulenga H. M. , , and Mason S. J. , 1999: Exploratory long-range models to estimate summer climate variability over southern Africa. J. Climate, 12, 18921899.

    • Search Google Scholar
    • Export Citation
  • Kirtman, B. P., , Shukla J. , , Huang B. , , Zhu Z. , , and Schneider E. K. , 1997: Multiseasonal predictions with a coupled ocean–global atmosphere system. Mon. Wea. Rev., 125, 789808.

    • Search Google Scholar
    • Export Citation
  • Klopper, E., , Landman W. A. , , and van Heerden J. , 1998: The predictability of seasonal maximum temperature in South Africa. Int. J. Climatol., 18, 741758.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , and Mason S. J. , 1999: Operational long-lead prediction of South African rainfall using canonical correlation analysis. Int. J. Climatol., 19, 10731090.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , and Mason S. J. , 2001: Forecasts or near-global sea surface temperatures using canonical correlation analysis. J. Climate, 14, 38193833.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , and Goddard L. , 2002: Statistical recalibration of GCM forecast over southern Africa using model output statistics. J. Climate, 15, 20382055.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , and Beraki A. , 2012: Multi-model forecast skill for mid-summer rainfall over southern Africa. Int. J. Climatol., 32, 303314.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , Mason S. J. , , Tyson P. D. , , and Tennant W. J. , 2001: Retro-active skill of multi-tiered forecasts of summer rainfall over southern Africa. Int. J. Climatol., 21, 119.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , Botes S. , , Goddard L. , , and Shongwe M. , 2005: Assessing the predictability of extreme rainfall seasons over southern Africa. Geophys. Res. Lett., 32, L23818, doi:10.1029/2005GL023965.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , Engelbrecht F. , , Beraki A. , , Engelbrecht C. , , Mbedzi M. , , Gill T. , , and Ntsangwane L. , 2009a: Model output statistics applied to multi-model ensemble long-range forecasts over South Africa. Water Research Commission Rep. 1492/1/08, 56 pp.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , Kgatuke M.-J. , , Mbedzi M. , , Beraki A. , , Bartman A. , , and du Piesanie A. , 2009b: Performance comparison of some dynamical and empirical downscaling methods for South Africa from a seasonal climate modelling perspective. Int. J. Climatol., 29, 15351549.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , DeWitt D. , , and Lee D.-E. , 2011: The high-resolution global SST forecast set of the CSIR. Proc. 27th Annual Conf. of South African Society for Atmospheric Sciences, Hartbeespoort, North-West Province, South Africa, Council for Scientific and Industrial Research, 39–40.

    • Search Google Scholar
    • Export Citation
  • Li, S., , Goddard L. , , and DeWitt D. G. , 2008: Predictive skill of AGCM seasonal climate forecasts subject to different SST prediction methodologies. J. Climate, 21, 21692186.

    • Search Google Scholar
    • Export Citation
  • Mason, S. J., 1995: Sea-surface temperature—South African rainfall associations, 1910–1989. Int. J. Climatol., 15, 119135.

  • Mason, S. J., 1998: Seasonal forecasting of South African rainfall using a non-linear discriminant analysis model. Int. J. Climatol., 18, 147164.

    • Search Google Scholar
    • Export Citation
  • Mason, S. J., , and Graham N. E. , 2002: Areas beneath the relative operating characteristics (ROC) and levels (ROL) curves: Statistical significance and interpretation. Quart. J. Roy. Meteor. Soc., 128, 21452166.

    • Search Google Scholar
    • Export Citation
  • McGregor, J. L., 2005: CCAM: Geometric aspects and dynamical formulation. CSIRO Tech. Paper 701, 44 pp.

  • McGregor, J. L., , Gordon H. B. , , Watterson I. G. , , Dix M. R. , , and Rotstayn L. D. , 1993: The CSIRO 9-level atmospheric general circulation model. CSIRO Division of Atmospheric Research Tech. Paper 26, 89 pp.

    • Search Google Scholar
    • Export Citation
  • Meehl, G. A., , Arblaster J. M. , , Lawrence D. M. , , Seth A. , , Schneider E. K. , , Kirtman B. P. , , and Min D. , 2006: Monsoon regimes in the CCSM3. J. Climate, 19, 24822495.

    • Search Google Scholar
    • Export Citation
  • Ndiaye, O., , Ward M. N. , , and Thiaw W. M. , 2011: Predictability of seasonal Sahel rainfall using GCMs and lead-time improvements through the use of a coupled model. J. Climate, 24, 19311949.

    • Search Google Scholar
    • Export Citation
  • Pacanowski, R. C., , and Griffies S. M. , 1998: MOM 3.0 manual. NOAA/Geophysical Fluid Dynamics Laboratory, 608 pp.

  • Palmer, T. N., and Coauthors, 2004: Development of a European Multimodel Ensemble System for Seasonal-to-Interannual Prediction (DEMETER). Bull. Amer. Meteor. Soc., 85, 853872.

    • Search Google Scholar
    • Export Citation
  • Pope, V. D., , Gallani M. , , Rowntree P. R. , , and Stratton R. A. , 2000: The impact of new physical parametrizations in the Hadley Centre climate model—HadAM3. Climate Dyn., 16, 123146.

    • Search Google Scholar
    • Export Citation
  • Power, S., , Haylock M. , , Colman R. , , and Wang X. , 2006: The predictability of interdecadal changes in ENSO activity and ENSO teleconnections. J. Climate, 19, 47554771.

    • Search Google Scholar
    • Export Citation
  • Reason, C. J. C., 1999: Interannual warm and cool events in the subtropical/mid-latitude south Indian Ocean. Geophys. Res. Lett., 26, 215218.

    • Search Google Scholar
    • Export Citation
  • Reason, C. J. C., , and Jagadheesha D. , 2005: Relationships between South Atlantic SST and atmospheric circulation over the South African region during austral winter. J. Climate, 18, 33393355.

    • Search Google Scholar
    • Export Citation
  • Reason, C. J. C., , and Rouault M. , 2005: Links between the Antarctic Oscillation and winter rainfall over western South Africa. Geophys. Res. Lett., 32, L07705, doi:10.1029/2005GL022419.

    • Search Google Scholar
    • Export Citation
  • Roeckner, E., and Coauthors, 1996: Simulation of present-day climate with the ECHAM4 model: Impact of model physics and resolution. Max Planck Institute for Meteorology Rep. 93, 171 pp.

    • Search Google Scholar
    • Export Citation
  • Rouault, M., , and Richard Y. , 2005: Intensity and spatial extent of droughts in southern Africa. Geophys. Res. Lett., 32, L15702, doi:10.1029/2005GL022436.

    • Search Google Scholar
    • Export Citation
  • Saha, S., and Coauthors, 2006: The NCEP Climate Forecast System. J. Climate, 19, 34833517.

  • Shongwe, M. E., , Landman W. A. , , and Mason S. J. , 2006: Performance of recalibration systems for GCM forecasts for southern Africa. Int. J. Climatol., 26, 15671585.

    • Search Google Scholar
    • Export Citation
  • Stockdale, T. N., , Anderson D. L. T. , , Alves J. O. S. , , and Balmaseda M. A. , 1998: Global seasonal rainfall forecasts using a coupled ocean–atmosphere model. Nature, 392, 370373.

    • Search Google Scholar
    • Export Citation
  • Tennant, W. J., , and Hewitson B. , 2002: Intra-seasonal rainfall characteristics and their importance to the seasonal prediction problem. Int. J. Climatol., 22, 10331048.

    • Search Google Scholar
    • Export Citation
  • Tippett, M. K., , Barnston A. G. , , and DeWitt D. , 2005: Statistical correction of tropical Pacific sea surface temperature forecasts. J. Climate, 18, 51415162.

    • Search Google Scholar
    • Export Citation
  • Troccoli, A., , Harrison M. , , Anderson D. L. T. , , and Mason S. J. , 2008: Seasonal Climate: Forecasting and Managing Risk. NATO Science Series on Earth and Environmental Sciences, Vol. 82, Springer, 467 pp.

    • Search Google Scholar
    • Export Citation
  • Van den Dool, H. M., 1994: Searching for analogues, how long must we wait? Tellus, 46A, 314324.

  • Van den Dool, H. M., , and Toth Z. , 1991: Why do forecasts for “near normal” often fail? Wea. Forecasting, 6, 7685.

  • Van Heerden, J., , Terblanche D. E. , , and Schulze G. C. , 1988: The Southern Oscillation and South African summer rainfall. Int. J. Climatol., 8, 577597.

    • Search Google Scholar
    • Export Citation
  • Van Rooy, M. P., 1972: Climate of South Africa: District rainfall for South Africa and the annual march of rainfall over southern Africa, part 10. South African Weather Bureau Tech. Note WB 35, Dept. of Transport, Pretoria, South Africa, 116 pp.

  • Washington, R., , and Preston A. , 2006: Extreme wet years over southern Africa: Role of the Indian Ocean sea surface temperatures. J. Geophys. Res., 111, D15104, doi:10.1029/2005JD006724.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. 2nd ed. Academic Press, 627 pp.

  • Yeh, S.-W., , Wu R. , , and Kirtman B. P. , 2007: Impact of the Indian Ocean on ENSO variability in a hybrid coupled model. Quart. J. Roy. Meteor. Soc., 133, 445457.

    • Search Google Scholar
    • Export Citation
  • View in gallery

    Differences in Kendall’s tau values between the 1-month lead-time CGCM-MOSslp (ECHAM4.5–MOM3-DC2) and the AGCM-MOSslp (ECHAM4.5 forced with statistically predicted SSTs) predicting DJF rainfall. The SLP area used as a predictor is located over the Indian Ocean between 10°–30°S and 45°–80°E. Positive Kendall’s tau differences indicate where the coupled model outperforms the atmospheric model. The black dots indicate the centroid positions of the 93 rainfall districts.

  • View in gallery

    (a) ROC scores obtained by retroactively predicting wet seasons (75th percentile values of the climatological record) probabilistically over 14 yr (1995/96–2008/09). The x axes show the 3-month rainfall seasons for which the forecasts are made, and the y axes show the forecast lead times in months. The ROC scores of the forecasts downscaled from (top) the ECHAM4.5 AGCM forced with statistically predicted SSTs, (middle) the ECHAM4.5-GML–CFSSST coupled model, and (bottom) the ECHAM4.5–MOM3-DCs coupled model. (b) As in (a), but for predicting dry seasons (25th percentile values of the climatological record).

  • View in gallery

    (a) ROC score differences between the probabilistic retroactive forecasts and AMIP-style probabilistic simulations of wet seasons (75th percentile values of the climatological record) over the 14 test years (1995/96–2008/09). The x axes show the 3-month rainfall seasons for which the forecasts and simulations are made, and the y axes show the forecast lead times in months. The ROC score differences between the simulations and the forecasts downscaled from (top) the ECHAM4.5 AGCM forced with statistically predicted SSTs, (middle) the ECHAM4.5-GML–CFSSST coupled model, and (bottom) the ECHAM4.5–MOM3-DCs coupled model. (b) As in (a), but for predicting and simulating dry seasons (25th percentile values of the climatological record).

  • View in gallery

    Reliability diagram and frequency histogram for above- (>75th percentile) and below- (<25th percentile) normal DJF rainfall 1-month lead forecasts produced by downscaling the ECHAM4.5–MOM3-DC2 coupled model to South African rainfall districts. The thick blue (red) curve and the blue (red) bars represent the wet (dry) category. The thin blue (red) line is the weighted least squares regression line of the wet (dry) reliability curve.

  • View in gallery

    Area-averaged observed DJF rainfall index (OBS) over South Africa vs retroactive forecasts (DC2) obtained by downscaling the ECHAM4.5–MOM3-DC2 1-month-lead predictions to district level. The years along the x axis refer to the Decembers in the DJF seasons. The association between the area-averaged forecasts and observed values is indicated by the Kendall’ tau value (significant at 95%) on the graph.

  • View in gallery

    Kendall’s tau values calculated between the observed and ECHAM4.5–MOM3-DC2 coupled model 1-month lead-time retroactive DJF rainfall predictions downscaled to district level. The confidence levels, calculated with a re-randomization or Monte Carlo test, for 90%, 95%, and 99% are, respectively, 0.27, 0.34, and 0.47 (corresponding to the color thresholds of the map).

  • View in gallery

    Area-averaged Kendall’s tau values for 27 (DJFall; 1982/83–2008/09), 13 (DJF1st; 1982/83–1994/95), and 14 (DJF2nd; 1995/96–2008/09) years of observed vs ECHAM4.5–MOM3-DC2 coupled model 1-month lead-time cross-validated DJF rainfall predictions downscaled to district level. One standard deviation error bars are shown as is the number of rainfall districts with Kendall’s tau values with local significance at the 95% level.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 366 366 4
PDF Downloads 93 93 10

Seasonal Rainfall Prediction Skill over South Africa: One- versus Two-Tiered Forecasting Systems

View More View Less
  • 1 Council for Scientific and Industrial Research, Natural Resources and the Environment, and Department of Geography, Geoinformatics and Meteorology, University of Pretoria, Pretoria, South Africa
  • | 2 International Research Institute for Climate and Society, Columbia University, Palisades, New York
  • | 3 South African Weather Service, Pretoria, South Africa
  • | 4 Council for Scientific and Industrial Research, Natural Resources and the Environment, Pretoria, South Africa
© Get Permissions
Full access

Abstract

Forecast performance by coupled ocean–atmosphere or one-tiered models predicting seasonal rainfall totals over South Africa is compared with forecasts produced by computationally less demanding two-tiered systems where prescribed sea surface temperature (SST) anomalies are used to force the atmospheric general circulation model. Two coupled models and one two-tiered model are considered here, and they are, respectively, the ECHAM4.5–version 3 of the Modular Ocean Model (MOM3-DC2), the ECHAM4.5-GML–NCEP Coupled Forecast System (CFSSST), and the ECHAM4.5 atmospheric model that is forced with SST anomalies predicted by a statistical model. The 850-hPa geopotential height fields of the three models are statistically downscaled to South African Weather Service district rainfall data by retroactively predicting 3-month seasonal rainfall totals over the 14-yr period from 1995/96 to 2008/09. Retroactive forecasts are produced for lead times of up to 4 months, and probabilistic forecast performance is evaluated for three categories with the outer two categories, respectively, defined by the 25th and 75th percentile values of the climatological record. The resulting forecast skill levels are also compared with skill levels obtained by downscaling forecasts produced by forcing the atmospheric model with simultaneously observed SST in order to produce a reference forecast set. Downscaled forecasts from the coupled systems generally outperform the downscaled forecasts from the two-tiered system, but neither of the two systems outscores the reference forecasts, suggesting that further improvement in operational seasonal rainfall forecast skill for South Africa is still achievable.

Corresponding author address: Willem A. Landman, Council for Scientific and Industrial Research, P.O. Box 395, Pretoria 0001, South Africa. E-mail: walandman@csir.co.za

Abstract

Forecast performance by coupled ocean–atmosphere or one-tiered models predicting seasonal rainfall totals over South Africa is compared with forecasts produced by computationally less demanding two-tiered systems where prescribed sea surface temperature (SST) anomalies are used to force the atmospheric general circulation model. Two coupled models and one two-tiered model are considered here, and they are, respectively, the ECHAM4.5–version 3 of the Modular Ocean Model (MOM3-DC2), the ECHAM4.5-GML–NCEP Coupled Forecast System (CFSSST), and the ECHAM4.5 atmospheric model that is forced with SST anomalies predicted by a statistical model. The 850-hPa geopotential height fields of the three models are statistically downscaled to South African Weather Service district rainfall data by retroactively predicting 3-month seasonal rainfall totals over the 14-yr period from 1995/96 to 2008/09. Retroactive forecasts are produced for lead times of up to 4 months, and probabilistic forecast performance is evaluated for three categories with the outer two categories, respectively, defined by the 25th and 75th percentile values of the climatological record. The resulting forecast skill levels are also compared with skill levels obtained by downscaling forecasts produced by forcing the atmospheric model with simultaneously observed SST in order to produce a reference forecast set. Downscaled forecasts from the coupled systems generally outperform the downscaled forecasts from the two-tiered system, but neither of the two systems outscores the reference forecasts, suggesting that further improvement in operational seasonal rainfall forecast skill for South Africa is still achievable.

Corresponding author address: Willem A. Landman, Council for Scientific and Industrial Research, P.O. Box 395, Pretoria 0001, South Africa. E-mail: walandman@csir.co.za

1. Introduction

The seasonal-to-interannual variability of rainfall and temperature anomalies over southern Africa is predictable (e.g., Klopper et al. 1998; Landman and Goddard 2002; Reason and Rouault 2005, Tennant and Hewitson 2002). This knowledge led to the development of objective operational seasonal forecasting systems for South Africa, but only as recently as the 1990s (e.g., Jury 1996; Jury et al. 1999; Landman and Mason 1999; Mason 1998). Although the prediction problem over southern Africa was also addressed by modelers from outside the region (e.g., Barnston et al. 1996), the South African–based institutions that started with prediction system development include the South African Weather Service (SAWS) and the Universities of the Witwatersrand, Pretoria, Cape Town, and Zululand. Although most of these efforts have initially been largely fragmented, the institutions in South Africa involved with the operational running of global circulation models started in 2003 to display their forecasts on the Global Forecasting Centre for Southern Africa web site (www.gfcsa.net). Since then, major advancements regarding seasonal forecast development have occurred at a number of institutions in South Africa, including perhaps most notably the SAWS’s designation as a Global Producing Centre for Long-Range Forecasts by the World Meteorological Organization. Since 2008, operational forecasts from local and international centers have been objectively combined into downscaled multimodel forecasts for the Southern African Development Community (SADC; Landman and Beraki 2012; Landman et al. 2009a).

Estimation of the evolution of SST anomalies, which are often relatively predictable, and subsequently employing them in atmospheric general circulation models (AGCMs), provides a means of generating forecasts of seasonal-average weather (Graham et al. 2000; Goddard and Mason 2002). Such a so-called two-tiered procedure to predicting the outcome of the rainfall season has been employed in South Africa for a number of years (e.g., Landman et al. 2001). The advent of fully coupled ocean–atmosphere models (e.g., Stockdale et al. 1998), or one-tiered systems, promised improved seasonal forecasts since in theory coupled general circulation models (CGCMs) should eventually outperform two-tiered systems because the former is able to describe the feedback between the ocean and atmosphere while the latter assumes that the atmosphere responds to SST but does not in turn affect the oceans (Copsey et al. 2006). This notion will be tested here by specifically focusing on the seasonal rainfall predictability over South Africa. If verification results to be presented here indicate that some of the current coupled models have at least caught up with AGCMs regarding operational forecast performance over the region, then South Africa, as one of the leading modeling countries in Africa, should direct more of their seasonal modeling efforts toward the use and even further development of coupled models for operational seasonal forecasting.

Coupled ocean–atmosphere models have been tested as seasonal forecasting systems since the 1990s (Barnett et al. 1993). Such coupled systems allow feedback between the predicted SST and the overlying atmosphere (Graham et al. 2005; Saha et al. 2006). However, predicting southern African climate variability using so-called two-tiered systems where only the atmosphere has been modeled while specifying the boundary conditions such as sea surface temperatures (SSTs), is an approach that has been used more extensively than coupled models both abroad (e.g., Goddard and Mason 2002) and locally (e.g., Reason and Jagadheesha 2005). One reason is that coupled systems effectively require twice the computer processing power than two-tiered systems do. Therefore, two-tiered systems are of particular importance to centers like SAWS, which has the infrastructure to run global atmospheric models operationally but which do not necessarily have the capabilities to run coupled models in real time. In fact, only a few of the global producing centers for long-range forecasts use these models for operational seasonal forecasting. Since it has been suggested that model performance may be a function of spatial resolution (Meehl et al. 2006; Hurrel et al. 2009) and ensemble size (Brown and Murphy 1996), other reasons for using two-tiered instead of coupled systems for operational forecasting include that the former system allows for the operational production of higher-resolution seasonal forecasts as well as larger ensemble forecasts since they are computationally less expensive than the latter. Moreover, when skillful SST forecasts are used, two-tiered systems may perform at least equally well as the current coupled systems (Troccoli et al. 2008). However, coupled models are widely acknowledged to represent the state of the art of seasonal forecasting, and it has been shown through the Development of a European Multimodel Ensemble System for Seasonal-to-Interannual Prediction (DEMETER) project (Palmer et al. 2004) that fully coupled systems can predict both the evolution of SSTs and atmospheric conditions at elevated levels of skill. In fact, southern African midsummer rainfall variability has been shown to be sufficiently predictable by using the coupled model output from the DEMETER project, especially during El Niño and La Niña seasons (Landman and Beraki 2012).

2. Data, models, and methods

a. Rainfall data

The district rainfall dataset of the South African Weather Service (Van Rooy 1972) is used to calculate 3-month seasonal rainfall totals for 93 evenly distributed locations across South Africa. The 3-month seasons considered are September–November (SON), October–December (OND), etc. through to February–April (FMA). This 8-month period covers the austral summer rainfall seasons over South Africa. This rainfall dataset consists of monthly values from 1921 to the present and the geographical description of the district boundaries are presented in Van Heerden et al. (1988). Figures presented in this paper only show the centroids of the districts in order not to clutter the figures.

b. The archived data of the general circulation models

The global model data used in this study are obtained from the data library of the International Research Institute for Climate and Society (IRI; http://iridl.ldeo.columbia.edu/). The AGCM data used are produced by the ECHAM4.5 (Roeckner et al. 1996) and consists of two sets. The first set (available from January 1950 to the present) is produced by forcing the ECHAM4.5 with observed SST and consists of 24 ensemble members. The second set (available from 1957 to July 2008), also consisting of 24 ensemble members, is produced by forcing the model with SST anomalies that are forecast using constructed analogs (Van den Dool 1994). Forecast data from two coupled models are also used and their ocean models are, respectively, the version 3 of the Modular Ocean Model (MOM3; Pacanowski and Griffies 1998) directly coupled to the ECHAM4.5 (DeWitt 2005), and a slab mixed layer (denoted ECHAM4.5-GML–CFSSST) for the tropical western Pacific, Indian, and Atlantic Oceans with prescribed SST forecasts from the National Centers for Environmental Prediction’s (NCEP) Coupled Forecast System (CFS; Saha et al. 2006) in the central and eastern tropical Pacific. Each of these coupled model forecast sets consists of 12 ensemble members, and the data are available from January 1982 to the present.

There are four forecast lead times considered here. For the two-tiered and the ECHAM4.5-GML–CFSSST systems, forecasts are produced near the beginning of the month, and for the ECHAM4.5–MOM3-DC2 system they are near the end of the month (the ECHAM4.5–MOM3-DC2 data cutoff has since been moved to the second of the month). The convention used here to describe the lead times is as follows. A 1-month lead time for the former two model systems implies that there are about 3 weeks from the issuance of the forecast to the beginning of the forecast season. For example, a 1-month lead-time forecast for the DJF season is produced at the beginning of November, 2-month lead-time forecasts in early October, 3-month lead-time forecasts in early September, and 4-month lead-time forecasts in early August. For the ECHAM4.5–MOM3-DC2 system used here, there are at least 4 weeks between the production of the forecast and the first month of the forecast season. For example, DJF forecasts at a 1-month lead time are produced near the end of October, 2-month lead-time forecasts at the end of September, 3-month lead-time forecasts at the end of August, and 4-month lead-time forecasts at the end of July. This convention implies that the forecast lead times of the ECHAM4.5–MOM3-DC2 system are slightly longer than the corresponding lead times of the other model systems.

c. Model output statistics

The general circulation models used here are configured with an effective horizontal resolution of 100–300 km. Being unable to represent local subgrid features, rainfall over southern Africa is often overestimated by models, but it has been demonstrated that such biases over southern Africa can be minimized through statistical postprocessing of the forecast data by improving on the raw rainfall output of the model (Landman and Goddard 2002; Landman et al. 2009b). Moreover, the postprocessing will also have as a result model forecast data directly applicable at a point of interest, such as at the centroids of the rainfall districts across South Africa employed here. Such empirical remapping of GCM fields to regional rainfall has already been successfully employed (e.g., Landman and Goddard 2002; Shongwe et al. 2006). Model output statistics (MOS) equations are developed here because they can compensate for systematic deficiencies in the global models directly in the regression equations (Wilks 2006) and have already been successfully employed in a study of AGCM versus coupled model performance (Ndiaye et al. 2011). Since MOS uses predictor values from the global models in both the development and forecast stages, these model errors are subsequently reduced. Still, the selection of the appropriate model field to be downscaled may require careful consideration. Candidate fields could include the raw model rainfall output, geopotential height fields and moisture field at standard pressure levels, thickness fields, etc. (Landman et al. 2001; Landman and Goddard 2002). Variables such as large-scale circulation are more accurately simulated by models than rainfall and should therefore be used instead in a MOS system to predict seasonal rainfall totals (Landman and Goddard 2002). The reason why model rainfall is not as accurately simulated by coarse-resolution models in which topography is poorly resolved is that rainfall fields are noisy, even when totaled over a season (Landman and Beraki 2012). As has been done before and is documented in the references above, the models’ 850-hPa geopotential height fields have been selected as predictors in the MOS equations.

The MOS equations are developed by using the canonical correlation analysis (CCA) option of the Climate Predictability Tool (CPT) of the IRI (http://iri.columbia.edu). The forecast fields from each global model used in the MOS are restricted over a domain that covers an area between the equator and 45°S and from 15°W to 60°E. Empirical orthogonal function (EOF) analysis is performed on both the predictor (the models’ 850-hPa geopotential height fields) and predictand sets (district rainfall—the option of the CPT is used that transforms the rainfall data into an approximate normal distribution) prior to CCA, and the number of EOF and CCA modes to be retained in the CPT’s CCA procedure is determined using cross-validation skill sensitivity tests. The predictor and predictand fields are first standardized, resulting in correlation matrices on which the EOF analysis is performed.

To minimize the artificial inflation of forecast skill, the downscaled forecast performance of the individual models should be verified over a test period that is independent of the training period and should involve evaluation of predictions compared to their matching observations excluding any information following the forecast year. Such a system mimics a true operational forecasting environment where no prior knowledge of the coming season is available. For DJF rainfall, the MOS models are first trained with information from 1982/83 and leading up to and including 1994/95, resulting in a first training set of 13 yr. The seasonal rainfall of the next year (1995/96) is subsequently predicted using the trained models. The various MOS models are subsequently retrained using information leading up to and including 1995/96, resulting in 14 yr on which the MOS equations are trained, to predict for 1996/97 conditions. This procedure is continued until the 2008/09 DJF rainfall is predicted using MOS systems trained with data from 1982/83 to 2007/08, resulting in 14 yr (1995/96–2008/09) of independent forecast data. The length of the training period may have an effect on the robustness or stability of the MOS equations. However, although forecast skill may not be constant in time, the dominant modes of variability included in the MOS equations should remain similar for a variety of training periods (Landman and Goddard 2002; Landman and Beraki 2012).

d. Verification

In estimating the skill in predicting seasonal rainfall totals over South Africa, the observed and predicted fields are separated into three categories defining above-normal, near-normal, and below-normal seasonal rainfall totals. However, these categories are not equiprobable here since the above- and below-normal threshold values, respectively, represent the 75th and 25th percentile values of the climatological record. The decision to use these thresholds instead of the usual equiprobable three-category design as defined by the 33rd and 67th percentile values of the climatological record is because users of seasonal forecast information may be more interested in the prediction of extreme seasons. Such a category description has been used before in verifying DEMETER models (Palmer et al. 2004) predicting Niño-3.4 SST (Troccoli et al. 2008), and it has also been shown that the prediction of extreme rainfall seasons over southern Africa has skill (Landman et al. 2005).

The distribution of individual ensemble members is supposed to be able to indicate forecast uncertainty. However, only a finite ensemble is available (12 or 24 members depending on the available global model data), suggesting that the forecast distribution may be poorly sampled and also differently sampled owing to the differences in the available ensemble sizes, and so the uncertainty associated with the forecasts has to be estimated. Probabilistic MOS forecasts for each of the 14 retroactive years are obtained here from the error variance of the cross-validated predictions using the ensemble mean (Troccoli et al. 2008) for each of the various training periods. To minimize the chance of obtaining biased results, cross validation is performed using a large 5-yr-out window, which means that 2 yr on either side of the predicted year are omitted. A new set of cross-validation forecasts is produced for each of the 14-yr forecasts since the cross-validation period is progressively increased by 1 yr at each forecast step.

Seasonal climate is inherently probabilistic, and so the district rainfall forecasts are also judged probabilistically. Two of the main attributes of interest for probabilistic forecasts are discrimination (are the forecasts discernibly different given different outcomes?) and reliability (is the confidence communicated in the forecast appropriate?). The forecast verification measures presented here for testing of the aforementioned attributes are the relative operating characteristic (ROC; Mason and Graham 2002) and the reliability diagram (Hamill 1997; Wilks 2006). ROC applied to probabilistic forecasts indicates whether the forecast probability was higher when an event such as a flood or drought season occurred compared to when it did not occur. ROC scores for the rainfall categories represent the respective areas beneath the ROC curve that are produced by plotting the forecast hit rates against the false alarm rates. If the area would be ≤0.5, the forecasts have no skill, and for a maximum ROC score of 1.0, perfect discrimination has been obtained. The forecasts are considered reliable if there is consistency between the predicted probabilities of the defined rainfall categories and the observed relative frequencies of the observed rainfall being assigned to these categories.

3. South African rainfall variability as a response to Indian Ocean coupling

SST anomalies of the Indian Ocean are related to southern African austral summer rainfall variability (e.g., Mason 1995). Anomalously warm SSTs in the central and tropical western Indian Ocean are often associated with drier than average conditions over southern Africa, a conclusion that has been supported by AGCM simulations (e.g., Goddard and Graham 1999). In addition to the link of the Indian Ocean to droughts, southern Africa has also experienced extremes of above-average seasonal rainfall: a specific pattern of SST in the southwestern Indian Ocean, with warm anomalies in the subtropical southwestern Indian Ocean and anomalously cold SSTs farther north, has been found to play a crucial role in causing extremely wet summer conditions over southern Africa (Washington and Preston 2006).

Over the past few decades the Indian Ocean has exhibited a rapid rise in SST. Rising SST should lead to decreasing sea level pressure (SLP) over the area of heating. But SLP increased in the Indian Ocean over the same period, and the increases in SLP overlie increases in SST (Copsey et al. 2006). However, it has been found that common atmospheric model errors with relation to decreasing modeled SLP can result when fixing the SST as a representation of ocean–atmosphere coupling in the Indian Ocean, suggesting that the response of some AGCMs to Indian Ocean warming may not provide a reliable guide to reality (Copsey et al. 2006), while air–sea coupling in the Indian Ocean is required for simulating Indian monsoon–ENSO relationships and for studying the influence of the Indian Ocean on ENSO variability (Yeh et al. 2007). In this paper we are comparing skill levels from AGCMs and coupled models, and so it would be of interest to see how these two systems simulate the relationship between Indian Ocean SLP and South African rainfall.

The SLP area considered is 10°–30°S, 45°–80°E (Copsey et al. 2006) and covers the larger part of the SST area of superimposed anomalies used in Washington and Preston (2006) cited above. Similar to how the 850-hPa heights over southern Africa are being used as the predictors in the MOS equations described above, for this part of the study the SLP fields over the above-mentioned Indian Ocean areas of, respectively, the ECHAM4.5 AGCM and the ECHAM4.5–MOM3-DC2 are used as predictors of DJF South African district rainfall. If the SLP of the coupled model is a better predictor of DJF rainfall than the SLP of the AGCM, then one could also assume that the coupled model should be better able to simulate the low-level circulation over the southwestern Indian Ocean that contributes to the country’s midsummer rainfall variability than the AGCM can and, hence, further justifies the notion that AGCM versus coupled model performance over South Africa should be investigated. Two sets of MOS equations, one each for the AGCM and coupled models, are subsequently developed for DJF rainfall forecasts produced at a 1-month lead time, and cross validated (5-yr-out window) over the 27-yr period from 1982/83 to 2008/09. The 5-yr-out design is long enough to minimize the effect trends in the data may have on forecast skill. MOS performance over the 27-yr period is described here in terms of the Kendall rank correlation coefficient, which is commonly referred to as Kendall’s tau (Wilks 2006). Kendall’s tau is a measure of rank correlation, and is considered a robust (to deviation from linearity) and resistant (to outlying data) alternative to Pearson’s or “ordinary” correlation. Figure 1 shows the difference in skill (Kendall’s tau) between the CGCM-MOSslp and the AGCM-MOSslp. Positive differences indicate where the coupled model outperforms the atmospheric model. Here, 75 of the 93 districts (81%) show positive Kendall’s tau differences, which suggests that the coupled model’s Indian Ocean SLP is a better predictor of South Africa’s midsummer rainfall variability than is the SLP of the atmospheric model. However, Monte Carlo testing for local significance (Wilks 2006) shows that only a small number of districts is associated with statistically significant differences: 17 of the 93 at the 90% level, 8 at the 95% level, and only 1 district at the 99% level show significantly different Kendall’s tau values.

Fig. 1.
Fig. 1.

Differences in Kendall’s tau values between the 1-month lead-time CGCM-MOSslp (ECHAM4.5–MOM3-DC2) and the AGCM-MOSslp (ECHAM4.5 forced with statistically predicted SSTs) predicting DJF rainfall. The SLP area used as a predictor is located over the Indian Ocean between 10°–30°S and 45°–80°E. Positive Kendall’s tau differences indicate where the coupled model outperforms the atmospheric model. The black dots indicate the centroid positions of the 93 rainfall districts.

Citation: Weather and Forecasting 27, 2; 10.1175/WAF-D-11-00078.1

Differences in Kendall’s tau are therefore small. Clearly, modeled Indian Ocean SLP as a predictor of South African midsummer rainfall is therefore not significantly sensitive to whether the SLP fields are obtained from the AGCM or from the coupled model. By using instead the two models’ 850-hPa geopotential height fields (equator–45°S and 15°W–60°E) as MOS predictors in a 27-yr, 5-yr-out cross-validation experiment, we show area-averaged Kendall’s tau values that are higher than those found from the MOS with SLP fields as predictors (for the AGCM, the improvement is from 0.15 to 0.25, and for the coupled model it is from 0.23 to 0.26). Moreover, the coupled model’s height fields as a midsummer rainfall predictor marginally outperform the AGCM equivalent. The improved performance of the models can be attributed to the fact that the predicting height fields are directly located over the subcontinent instead of some distance away from the continent, as is the case with the Indian Ocean SLP predictors. In addition, the main source of the predictable signal over southern Africa as captured in global models originates from the equatorial Pacific Ocean (Landman and Beraki 2012). Coupled models seem to emphasize the significance of the equatorial Pacific Ocean as the main source of South African rainfall predictability, and considering the results presented here that modeled Indian Ocean SLP seems to play a secondary role in the prediction of South African midsummer rainfall, the models’ ability to predict the seasonal rainfall seem likely to be remotely forced, possibly by the equatorial Pacific. The Pacific Ocean signal may be translated through the Indian Ocean, but this notion is not strongly supported by the modeling results presented here even though the extent and intensity of drought over southern Africa over the past 30–40 yr is possibly associated with a reinforced influence of El Niño through warmer tropical and subtropical Indian Ocean SSTs (Reason 1999; Rouault and Richard 2005). The mechanism through which the signal from the Pacific Ocean, or from any other location, is modeled to arrive over South Africa lies outside the scope of this paper. The results presented thus far suggest that there is a need to at least find out how the performance of an AGCM as a rainfall predictor for South Africa compares with that of coupled models, since both coupled and uncoupled approaches could be used (Jha and Kumar 2009) to investigate the observed coupled system over the region. The probabilistic forecast performance of the two systems (AGCM and CGCM) will be compared later in this paper in terms of their ability to discriminate between wet and dry seasons and not only on their respective associations with observed seasonal-to-interannual variability, as was done in this section.

4. Retroactive forecast skill

The following sections will outline the findings when verifying the three sets of MOS forecasts of the various model systems over the 14-yr retroactive period from 1995/96 to 2008/09.

a. Relative operating characteristics

Only the verification results for the above- and below-normal categories are presented here since there is usually little skill to be derived from predicting the near-normal category (Van den Dool and Toth 1991). In fact, ROC scores for the near-normal category, notwithstanding the fact that this category is defined here to contain 50% of the climatological record, are less or around 0.5 for all models, lead times, and seasons (not shown). Figure 2 shows the ROC scores obtained by retroactively predicting (a) wet and (b) dry seasons over the 14 retroactive years (1995/96–2008/09). Most of the predictability is found during the middle of the forecast period when tropical influences start to dominate the atmospheric circulation across South Africa, with almost no predictability evident during austral spring when the seasonal rainfall of South Africa is mostly influenced by transient weather systems. The seasonal forecast verification statistics of the IRI are displayed on their Climate Forecast Verification pages (http://iri.columbia.edu). These statistics are based, at the time of the writing of this paper, entirely on AGCM forecasts including ECHAM4.5 forecasts, and they show that the season of highest rainfall predictability over South Africa is found during NDJ. This peak in prediction skill is also seen in the top panel of Fig. 2 (ECHAM4.5 AGCM skill), giving support to the verification result presented here. However, the coupled models, especially the ECHAM4.5–MOM3-DC2 model, provide additional predictability during DJF (Fig. 2). All of the models are associated with maximum skill at the shortest lead time.

Fig. 2.
Fig. 2.

(a) ROC scores obtained by retroactively predicting wet seasons (75th percentile values of the climatological record) probabilistically over 14 yr (1995/96–2008/09). The x axes show the 3-month rainfall seasons for which the forecasts are made, and the y axes show the forecast lead times in months. The ROC scores of the forecasts downscaled from (top) the ECHAM4.5 AGCM forced with statistically predicted SSTs, (middle) the ECHAM4.5-GML–CFSSST coupled model, and (bottom) the ECHAM4.5–MOM3-DCs coupled model. (b) As in (a), but for predicting dry seasons (25th percentile values of the climatological record).

Citation: Weather and Forecasting 27, 2; 10.1175/WAF-D-11-00078.1

Higher skill is found here in predicting for wet seasons as opposed to dry seasons. A possible explanation for the higher skill during wet seasons follows. Forecast models for southern Africa seem to be most skillful during ENSO years and are particularly skillful at predicting above-normal rainfall totals during La Niña seasons (Landman and Beraki 2012). During the 14-yr test period, at the end of each of the predicted calendar years, the Oceanic Niño Index (ONI) was <−0.5 for 7 of the 14 yr, and >+0.5 for only 4 of the 14 yr. There were therefore more years during which the DJF Niño-3.4 index suggested cold event years than warm event years, and cold event years are also more likely to be associated with observed wet conditions over southern Africa as opposed to warm event years associated with observed dry seasons over southern Africa, similar to what has been found for Australian rainfall variability (Power et al. 2006).

The ROC score panels in Fig. 2 show that, in general, the coupled models outperform the AGCM over South Africa. The superior performance of the coupled systems is further demonstrated by the Brier skill score (BSS) and ranked probability skill score (RPSS) (Wilks 2006) as calculated over the 14-yr test period and given as percentages, with both scores based on climatology as the reference forecast: at a 1-month lead time, the BSS value for the above- (below-) normal category is 18.0 (−1.3) for the ECHAM4.5–MOM3-DC2 system, while the BSS value for the above- (below-) normal category is −0.8 (−5.7) for the ECHAM4.5 AGCM. The RPSS values for the CGCM and AGCM are, respectively, 7.3 and −2.1. These results are especially meaningful given the smaller ensemble size (12 versus 24) of the coupled models and the slightly longer lead times (±5 weeks versus ±3 weeks) associated with the ECHAM4.5–MOM3-DC2. Although the coupled model hindcasts are generally better able to discriminate wet and dry seasons, the ROC differences are not very large (about 0.1–0.15 for wet seasons during midsummer) suggesting that both systems are providing useful forecast information. A best practice may then be to utilize the attributes from both systems in a multimodel ensemble approach for operational seasonal forecasting purposes (Li et al. 2008).

The reason for the difference in skill between the CGCMs and AGCM could be attributed to the skill of the SST forecasts forcing the AGCM. How would the coupled models’ skill, in terms of discriminating wet–dry seasons from the rest, compare with the skill of the AGCM given perfectly predicted SSTs? This question is addressed by assuming that it would be possible to predict the forcing SST perfectly over the 4-month lead times considered here. Figure 3 shows ROC score differences between the three forecast systems and Atmospheric Model Intercomparison Project (AMIP) style simulations (Gates 1992) of the ECHAM4.5 AGCM over the six 3-month seasons considered. Predicting for wet seasons, the AGCM would have been able to produce higher ROC scores than the scores found when forcing the AGCM with constructed analog SSTs, and given perfect SSTs, the AGCM would outscore the coupled models too. For dry seasons, however, most of the advantage of perfectly predicted SST is found only during the DJF season. Therefore, with improved SST forecasts, the ECHAM4.5 AGCM could perform at least as well as its coupled versions. The use of AGCMs as an operational forecasting tool for South African rainfall variability is likely to continue for quite a while longer, because they are cheaper to run than coupled models and can, by utilizing the same available computational resources, be used for higher-resolution runs and for the generation of more ensemble members than would be the case for coupled models. In South Africa, some modeling effort is being directed toward the development of improved global SST forecasts through multimodel ensembles (Landman et el. 2011) for the purpose of forcing locally run AGCMs for operational forecast production. The current SST forecasts are presented as maps on the web site of the South African Risk and Vulnerability Atlas (SARVA; http://rava.qsens.net/themes/climate_template/seasonal-forecasts), as output files on the servers of the Council for Scientific and Industrial Research (CSIR; ftp.csir.co.za), and are already being used for operational forecast production at the University of Cape Town in South Africa (http://www.gfcsa.net/CSAG/fcstSSTs/) and at the South African Weather Service.

Fig. 3.
Fig. 3.

(a) ROC score differences between the probabilistic retroactive forecasts and AMIP-style probabilistic simulations of wet seasons (75th percentile values of the climatological record) over the 14 test years (1995/96–2008/09). The x axes show the 3-month rainfall seasons for which the forecasts and simulations are made, and the y axes show the forecast lead times in months. The ROC score differences between the simulations and the forecasts downscaled from (top) the ECHAM4.5 AGCM forced with statistically predicted SSTs, (middle) the ECHAM4.5-GML–CFSSST coupled model, and (bottom) the ECHAM4.5–MOM3-DCs coupled model. (b) As in (a), but for predicting and simulating dry seasons (25th percentile values of the climatological record).

Citation: Weather and Forecasting 27, 2; 10.1175/WAF-D-11-00078.1

b. Reliability

In general, the highest ROC scores are obtained from the downscaled forecasts of the ECHAM4.5–MOM3-DC2 coupled model, and DJF is the season of highest predictability. The remainder of the verification discussion will be for this model and season. Figure 4 shows the reliability diagram at a 1-month lead time (i.e., forecasts produced near the end of October for South African DJF rainfall totals). In addition to the reliability curves for the 25th (drought) and 75th (wet) percentile thresholds, weighted least squares regression lines for the two categories are also presented. The weighting is relative to how frequently forecasts are issued at a given confidence level. Regression lines along the diagonal of the reliability diagram imply perfect reliability, while regression lines above (below) the diagonal imply that observed wet–dry DJF rainfall seasons tend to occur more (less) frequently than predicted. The most common slope of the weighted regression lines found for seasonal forecasting are shallower than the diagonal line (Landman and Beraki 2012; Troccoli et al. 2008), implying that seasonal forecasts are generally overconfident (a regression line steeper than the diagonal indicates underconfidence). The histograms in the reliability diagram of Fig. 4 show the frequencies with which wet–dry forecasts occur in probability intervals of 10%, starting at 5%, and reveals how strongly and frequently the issued forecast probabilities depart from the climatological probabilities. Dry (wet) seasons are associated here with the bottom (top) 25% of climatology.

Fig. 4.
Fig. 4.

Reliability diagram and frequency histogram for above- (>75th percentile) and below- (<25th percentile) normal DJF rainfall 1-month lead forecasts produced by downscaling the ECHAM4.5–MOM3-DC2 coupled model to South African rainfall districts. The thick blue (red) curve and the blue (red) bars represent the wet (dry) category. The thin blue (red) line is the weighted least squares regression line of the wet (dry) reliability curve.

Citation: Weather and Forecasting 27, 2; 10.1175/WAF-D-11-00078.1

The reliability of the DJF forecasts as revealed by Fig. 4 indicates that when predicting for wet conditions, the forecasts are generally underconfident, although a high level of reliability is indicated since the slope of the wet season regression line is close to the slope of perfect reliability. The frequency histogram further suggests some sharpness (forecasts that rarely deviate from the climatological value–25% here–exhibit no sharpness) for the prediction of wet conditions since the most frequent forecast category is between 25% and 35%. The second highest frequency is between 15% and 25%, suggesting that the forecasts for wet conditions still lack sharpness. For dry seasons, on the other hand, forecasts are overconfident, more typical of seasonal forecasts, and the forecast probabilities generally lack sharpness.

c. Deterministic skill assessment

The previous section presented verification statistics on the 14 retroactive years of probabilistic forecasts. Although seasonal forecasts issued operationally are for the most part of a probabilistic nature, it is often informative to investigate deterministic forecast performance. Here, the same 14 yr are used as before. Figure 5 shows the area-averaged and then normalized observed DJF rainfall index over South Africa, versus 1-month lead-time retroactive forecasts obtained by downscaling the ECHAM4.5–MOM3-DC2 predictions to South African district level. Three of the four driest years, as a spatial average, are associated with El Niño events (1997/98, 2002/03, and 2006/07) and three of the four wettest years with La Niña events (1995/96, 1999/00, and 2007/08). The signs of the rainfall indices during these 6 yr were captured by the forecasts, emphasizing the already demonstrated skill found during ENSO years for South African midsummer rainfall variability, while the large error seen for the 2001/02 season may be attributed to the fact that most models fare poorly when predicting southern African midsummer rainfall during ENSO-neutral years (Landman and Beraki 2012). The ECHAM–MOM3-DC2 forecast system predicted the sign of the rainfall anomaly correctly for these ENSO years, but not the size, especially for the 1997/98 season, when the model exaggerated the negative anomaly. This forecast error has also been found with forecast systems other than those presented here (Landman and Beraki 2012). Notwithstanding these discrepancies, the short-term rainfall “trends” from one midsummer season to the next are captured for 11 out of the 13 cases (the trends of 1998/99–1999/00 and 2003/04–2004/05 being the exceptions). This is a hit rate of 85% and suggests that the prediction of predictands other than seasonal rainfall totals could supplement existing forecast output disseminated to the end users of forecast information. For this example, users may be provided with the likelihood of an imminent DJF season to be wetter or drier than the DJF of the previous year.

Fig. 5.
Fig. 5.

Area-averaged observed DJF rainfall index (OBS) over South Africa vs retroactive forecasts (DC2) obtained by downscaling the ECHAM4.5–MOM3-DC2 1-month-lead predictions to district level. The years along the x axis refer to the Decembers in the DJF seasons. The association between the area-averaged forecasts and observed values is indicated by the Kendall’ tau value (significant at 95%) on the graph.

Citation: Weather and Forecasting 27, 2; 10.1175/WAF-D-11-00078.1

The spatial distribution of deterministic skill (again using Kendall’s tau) for predicting DJF rainfall at a 1-month lead time with the retroactively downscaled forecasts of the ECHAM4.5–MOM3-DC2 coupled model is shown in Fig. 6. The shading on the map represents calculated confidence levels using a re-randomization or Monte Carlo test (Wilks 2006) for 90%, 95%, and 99%, respectively. The threshold values for these confidence levels are respectively 0.27, 0.34, and 0.47. Highest skill is found over the northeastern interior of South Africa, the east coast, and parts of the central and western interior. This pattern of skill is not uncommon for South Africa and also has been found over various verification periods and parameters (Landman et al. 2009b). Notwithstanding, one may have expected higher skill estimates given that the coupled model used here is outscoring the AGCM forecasts on which South African rainfall prediction skill is most often based. However, as can be seen in Fig. 7, which shows 5-yr-out cross-validation results, higher skill is evident when considering the whole of the 27-yr period for which the raw GCM data are available for this study. Figure 7 shows that the cross-validated average Kendall’s tau value for the whole 27-yr period is higher than both the average values over the initial 13-yr cross-validation period on which the first MOS equations are based and the cross-validated average value obtained over the retroactive test period. This result indicates that the skill levels presented here over the 14-yr retroactive test period is in all likelihood an underestimation.

Fig. 6.
Fig. 6.

Kendall’s tau values calculated between the observed and ECHAM4.5–MOM3-DC2 coupled model 1-month lead-time retroactive DJF rainfall predictions downscaled to district level. The confidence levels, calculated with a re-randomization or Monte Carlo test, for 90%, 95%, and 99% are, respectively, 0.27, 0.34, and 0.47 (corresponding to the color thresholds of the map).

Citation: Weather and Forecasting 27, 2; 10.1175/WAF-D-11-00078.1

Fig. 7.
Fig. 7.

Area-averaged Kendall’s tau values for 27 (DJFall; 1982/83–2008/09), 13 (DJF1st; 1982/83–1994/95), and 14 (DJF2nd; 1995/96–2008/09) years of observed vs ECHAM4.5–MOM3-DC2 coupled model 1-month lead-time cross-validated DJF rainfall predictions downscaled to district level. One standard deviation error bars are shown as is the number of rainfall districts with Kendall’s tau values with local significance at the 95% level.

Citation: Weather and Forecasting 27, 2; 10.1175/WAF-D-11-00078.1

5. Discussion and conclusions

The South African modeling community has over the past decade or so expended a large amount of resources to establish the use of AGCMs as operational seasonal forecast tools. These models have all been developed outside of South Africa, but have been used extensively for operational seasonal forecast production as well as for research by institutions such as the South African Weather Service [Center for Ocean–Land–Atmosphere Studies (COLA T30), Kirtman et al. (1997); ECHAM4.5, Roeckner et al. (1996)], the Universities of Cape Town [the third climate configuration of the Met Office Unified Model (HadAM3P), Pope et al. (2000)] and Pretoria [Commonwealth Scientific and Industrial Research Organisation (CSIRO), McGregor et al. (1993); Conformal Cubic Atmospheric Model (CCAM), McGregor (2005)], and the Council for Scientific and Industrial Research (CCAM; McGregor 2005). Recently, South African modelers have also become involved with model development (Engelbrecht et al. 2007) and the combination of forecasts from various sources into multimodel systems (Landman and Beraki 2012). The establishment of a large computing infrastructures in South Africa, such as the Centre for High Performance Computing (CHPC; http://www.chpc.ac.za/), the supercomputer capabilities of the South African Weather Service, and computer clusters at local universities and at the Council for Scientific and Industrial Research, are all making it possible to further advance the modeling of the physical ocean–atmosphere–land–ice system. Moreover, in collaboration with international modeling partners, the South African modeling community is certainly geared to continue to make advances in the development of forecast models (e.g., improved cumulus parameterization) and systems (e.g., multimodel ensembles). For both the use and for the further development of models, a question to be addressed by modelers is whether or not available resources in South Africa should be directed more toward ocean–atmosphere models as opposed to AGCMs only. If coupled GCMs are beginning to outperform AGCMs, then there is good reason to use the forecasts from these models for operational work and even to invest in their implementation on local machines, similar to what has been done with AGCMs in the past. If the coupled models are lagging behind, then there is a need to further improve on these models since in theory coupled models should eventually outperform two-tiered systems because the former are able to describe the feedback between the ocean and atmosphere while the latter assume that the atmosphere responds to SST but does not in turn affect the oceans.

The main objective of this paper was to investigate how the operational forecast performance of an AGCM would compare with the performances of its ocean–atmosphere coupled versions. First, it was shown that both modeling systems produce about equally skillful forecasts when low-level Indian Ocean circulation is used as a predictor of midsummer rainfall over South Africa. This result suggests that the ocean–atmosphere coupling over the Indian Ocean may not really help to improve on operational seasonal rainfall forecasts for South Africa. Furthermore, modeled low-level circulation localized over the subcontinent improved on the forecasts that are based on Indian Ocean SLP, suggesting that the rainfall variability of South Africa may be remotely forced. To adequately answer questions pertaining to our understanding of the mechanisms responsible for southern Africa’s unique climate variability, one may need to use the attributes of both AGCMs and coupled models. Finding out how these modeling systems compare with each other could then be seen as a first step toward an improved understanding of how things work.

Verification of the two systems over a 14-yr test period that mimics a set of operational forecasts has shown that the coupled models outscored the AGCMs in terms of their ability to discriminate extreme (25th and 75th percentile thresholds) rainfall seasons from the rest. For the test period, all the forecasts are also found to be much more skillful (in terms of discrimination) during wet seasons as opposed to dry seasons, and that most of the predictability resides during midsummer when tropical influences start to dominate the atmospheric circulation. In terms of reliability, wet seasons could be predicted with confidence, but that the forecast probabilities do lack sharpness to some extent. For drought prediction, the forecasts are generally too confident and lack sharpness. None of the systems could, however, provide skillful predictions during spring, which makes seasonal onset forecasting very challenging, notwithstanding requests from South African farmers to provide such information.

Although the skill levels presented here are likely to be an underestimation, further improvement in skill is possible by forcing the AGCM with improved SSTs since it was found that perfectly “predicted” SSTs improved on the AGCM forecasts that subsequently outscored the coupled model forecasts, especially during the period of maximum skill (midsummer). SST forecasts can be improved by using and/or combining forecasts produced by statistical models (e.g., Landman and Mason 2001) and through postprocessing of coupled models (Tippett et al. 2005). In South Africa, the operational production of global SST forecasts for the purpose of helping to optimize operational AGCM forecasts is already under way, and the operationally produced SST are currently employed in local AGCM forecast production (e.g., http://www.gfcsa.net/CSAG/fcstSSTs/).

The geographical distribution of skill (maximum skill over the northeastern and central-western interior, etc.) and the forecast errors made by other models (e.g., the large forecast error made for the 1997/98 rainfall season) have been similar to what has been presented here, so this paper has not really contributed significantly to improving on the skill of South African seasonal rainfall prediction. What this paper has been successful in is demonstrating the ability of coupled models to simulate South African rainfall variability and that their atmosphere-only component can produce forecasts that are just as skillful on the condition that the AGCM is supplied with highly skillful SST forecasts. The latter is again possible through the utilization of SST forecasts from coupled models, which shows that AGCM forecasts will benefit from skillful coupled models. The South African modeling community will therefore have to expend at least part of their future resources on the implementation, use, and further development of coupled models and systems.

Acknowledgments

This material is based upon work partly supported financially by the National Research Foundation (NRF) of South Africa. The computing time used to produce the retrospective forecasts from IRI was largely provided by grants from the multiagency Climate Simulation Laboratory (CSL) program, with Dave DeWitt as PI.

REFERENCES

  • Barnett, T. P., , Latif M. , , Graham N. E. , , Flugel M. , , Pazan S. , , and White W. B. , 1993: ENSO and ENSO-related predictability. Part I: Prediction of equatorial Pacific sea surface temperature with a hybrid coupled ocean–atmosphere model. J. Climate, 6, 15451566.

    • Search Google Scholar
    • Export Citation
  • Barnston, A. G., , Thiao W. , , and Kumar V. , 1996: Long-lead forecasts of seasonal precipitation in Africa using CCA. Wea. Forecasting, 11, 506520.

    • Search Google Scholar
    • Export Citation
  • Brown, B. H., , and Murphy A. H. , 1996: Improving forecasting performance by combining forecasts: The example of road-surface temperature forecasts. Meteor. Appl., 3, 257265.

    • Search Google Scholar
    • Export Citation
  • Copsey, D., , Sutton R. , , and Knight J. R. , 2006: Recent trends in sea level pressure in the Indian Ocean region. Geophys. Res. Lett., 33, L19712, doi:10.1029/2006GL027175.

    • Search Google Scholar
    • Export Citation
  • DeWitt, D. G., 2005: Retrospective forecasts of interannual sea surface temperature anomalies from 1982 to present using a directly coupled atmosphere–ocean general circulation model. Mon. Wea. Rev., 133, 29722995.

    • Search Google Scholar
    • Export Citation
  • Engelbrecht, F. A., , McGregor J. L. , , and Rautenbach C. J.deW. , 2007: On the development of a new nonhydrostatic atmospheric model in South Africa. S. Afr. J. Sci., 103, 127134.

    • Search Google Scholar
    • Export Citation
  • Gates, W. L., 1992: AMIP: The Atmospheric Model Intercomparison Project. Bull. Amer. Meteor. Soc., 73, 19621970.

  • Goddard, L., , and Graham N. E. , 1999: Importance of the Indian Ocean for simulating rainfall anomalies over eastern and southern Africa. J. Geophys. Res., 104 (D16), 19 09919 116.

    • Search Google Scholar
    • Export Citation
  • Goddard, L., , and Mason S. J. , 2002: Sensitivity of seasonal climate forecasts to persisted SST anomalies. Climate Dyn., 19, 619631.

  • Graham, R. J., , Evans A. D. L. , , Milne K. R. , , Harrison M. S. J. , , and Robertson K. B. , 2000: As assessment of seasonal predictability using atmospheric general circulation models. Quart. J. Roy. Meteor. Soc., 126, 22112240.

    • Search Google Scholar
    • Export Citation
  • Graham, R. J., , Gordon M. , , McLean P. J. , , Ineson S. , , Huddleston M. R. , , Davey M. K. , , Brookshaw A. , , and Barnes R. T. H. , 2005: A performance comparison of coupled and uncoupled versions of the Met Office seasonal prediction general circulation model. Tellus, 57A, 320319.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 1997: Reliability diagrams for multicategory probabilistic forecasts. Wea. Forecasting, 12, 736741.

  • Hurrel, J., , Meehl G. A. , , Bader D. , , Delworth T. L. , , Kirtman B. , , and Wielicki B. , 2009: A unified modeling approach to climate system prediction. Bull. Amer. Meteor. Soc., 90, 18191832.

    • Search Google Scholar
    • Export Citation
  • Jha, B., , and Kumar A. , 2009: A comparison of the atmospheric response to ENSO in coupled and uncoupled model simulations. Mon. Wea. Rev., 137, 479487.

    • Search Google Scholar
    • Export Citation
  • Jury, M. R., 1996: Regional teleconnection patterns associated with summer rainfall over South Africa, Namibia and Botswana. Int. J. Climatol., 16, 135153.

    • Search Google Scholar
    • Export Citation
  • Jury, M. R., , Mulenga H. M. , , and Mason S. J. , 1999: Exploratory long-range models to estimate summer climate variability over southern Africa. J. Climate, 12, 18921899.

    • Search Google Scholar
    • Export Citation
  • Kirtman, B. P., , Shukla J. , , Huang B. , , Zhu Z. , , and Schneider E. K. , 1997: Multiseasonal predictions with a coupled ocean–global atmosphere system. Mon. Wea. Rev., 125, 789808.

    • Search Google Scholar
    • Export Citation
  • Klopper, E., , Landman W. A. , , and van Heerden J. , 1998: The predictability of seasonal maximum temperature in South Africa. Int. J. Climatol., 18, 741758.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , and Mason S. J. , 1999: Operational long-lead prediction of South African rainfall using canonical correlation analysis. Int. J. Climatol., 19, 10731090.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , and Mason S. J. , 2001: Forecasts or near-global sea surface temperatures using canonical correlation analysis. J. Climate, 14, 38193833.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , and Goddard L. , 2002: Statistical recalibration of GCM forecast over southern Africa using model output statistics. J. Climate, 15, 20382055.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , and Beraki A. , 2012: Multi-model forecast skill for mid-summer rainfall over southern Africa. Int. J. Climatol., 32, 303314.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , Mason S. J. , , Tyson P. D. , , and Tennant W. J. , 2001: Retro-active skill of multi-tiered forecasts of summer rainfall over southern Africa. Int. J. Climatol., 21, 119.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , Botes S. , , Goddard L. , , and Shongwe M. , 2005: Assessing the predictability of extreme rainfall seasons over southern Africa. Geophys. Res. Lett., 32, L23818, doi:10.1029/2005GL023965.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , Engelbrecht F. , , Beraki A. , , Engelbrecht C. , , Mbedzi M. , , Gill T. , , and Ntsangwane L. , 2009a: Model output statistics applied to multi-model ensemble long-range forecasts over South Africa. Water Research Commission Rep. 1492/1/08, 56 pp.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , Kgatuke M.-J. , , Mbedzi M. , , Beraki A. , , Bartman A. , , and du Piesanie A. , 2009b: Performance comparison of some dynamical and empirical downscaling methods for South Africa from a seasonal climate modelling perspective. Int. J. Climatol., 29, 15351549.

    • Search Google Scholar
    • Export Citation
  • Landman, W. A., , DeWitt D. , , and Lee D.-E. , 2011: The high-resolution global SST forecast set of the CSIR. Proc. 27th Annual Conf. of South African Society for Atmospheric Sciences, Hartbeespoort, North-West Province, South Africa, Council for Scientific and Industrial Research, 39–40.

    • Search Google Scholar
    • Export Citation
  • Li, S., , Goddard L. , , and DeWitt D. G. , 2008: Predictive skill of AGCM seasonal climate forecasts subject to different SST prediction methodologies. J. Climate, 21, 21692186.

    • Search Google Scholar
    • Export Citation
  • Mason, S. J., 1995: Sea-surface temperature—South African rainfall associations, 1910–1989. Int. J. Climatol., 15, 119135.

  • Mason, S. J., 1998: Seasonal forecasting of South African rainfall using a non-linear discriminant analysis model. Int. J. Climatol., 18, 147164.

    • Search Google Scholar
    • Export Citation
  • Mason, S. J., , and Graham N. E. , 2002: Areas beneath the relative operating characteristics (ROC) and levels (ROL) curves: Statistical significance and interpretation. Quart. J. Roy. Meteor. Soc., 128, 21452166.

    • Search Google Scholar
    • Export Citation
  • McGregor, J. L., 2005: CCAM: Geometric aspects and dynamical formulation. CSIRO Tech. Paper 701, 44 pp.

  • McGregor, J. L., , Gordon H. B. , , Watterson I. G. , , Dix M. R. , , and Rotstayn L. D. , 1993: The CSIRO 9-level atmospheric general circulation model. CSIRO Division of Atmospheric Research Tech. Paper 26, 89 pp.

    • Search Google Scholar
    • Export Citation
  • Meehl, G. A., , Arblaster J. M. , , Lawrence D. M. , , Seth A. , , Schneider E. K. , , Kirtman B. P. , , and Min D. , 2006: Monsoon regimes in the CCSM3. J. Climate, 19, 24822495.

    • Search Google Scholar
    • Export Citation
  • Ndiaye, O., , Ward M. N. , , and Thiaw W. M. , 2011: Predictability of seasonal Sahel rainfall using GCMs and lead-time improvements through the use of a coupled model. J. Climate, 24, 19311949.

    • Search Google Scholar
    • Export Citation
  • Pacanowski, R. C., , and Griffies S. M. , 1998: MOM 3.0 manual. NOAA/Geophysical Fluid Dynamics Laboratory, 608 pp.

  • Palmer, T. N., and Coauthors, 2004: Development of a European Multimodel Ensemble System for Seasonal-to-Interannual Prediction (DEMETER). Bull. Amer. Meteor. Soc., 85, 853872.

    • Search Google Scholar
    • Export Citation
  • Pope, V. D., , Gallani M. , , Rowntree P. R. , , and Stratton R. A. , 2000: The impact of new physical parametrizations in the Hadley Centre climate model—HadAM3. Climate Dyn., 16, 123146.

    • Search Google Scholar
    • Export Citation
  • Power, S., , Haylock M. , , Colman R. , , and Wang X. , 2006: The predictability of interdecadal changes in ENSO activity and ENSO teleconnections. J. Climate, 19, 47554771.

    • Search Google Scholar
    • Export Citation
  • Reason, C. J. C., 1999: Interannual warm and cool events in the subtropical/mid-latitude south Indian Ocean. Geophys. Res. Lett., 26, 215218.

    • Search Google Scholar
    • Export Citation
  • Reason, C. J. C., , and Jagadheesha D. , 2005: Relationships between South Atlantic SST and atmospheric circulation over the South African region during austral winter. J. Climate, 18, 33393355.

    • Search Google Scholar
    • Export Citation
  • Reason, C. J. C., , and Rouault M. , 2005: Links between the Antarctic Oscillation and winter rainfall over western South Africa. Geophys. Res. Lett., 32, L07705, doi:10.1029/2005GL022419.

    • Search Google Scholar
    • Export Citation
  • Roeckner, E., and Coauthors, 1996: Simulation of present-day climate with the ECHAM4 model: Impact of model physics and resolution. Max Planck Institute for Meteorology Rep. 93, 171 pp.

    • Search Google Scholar
    • Export Citation
  • Rouault, M., , and Richard Y. , 2005: Intensity and spatial extent of droughts in southern Africa. Geophys. Res. Lett., 32, L15702, doi:10.1029/2005GL022436.

    • Search Google Scholar
    • Export Citation
  • Saha, S., and Coauthors, 2006: The NCEP Climate Forecast System. J. Climate, 19, 34833517.

  • Shongwe, M. E., , Landman W. A. , , and Mason S. J. , 2006: Performance of recalibration systems for GCM forecasts for southern Africa. Int. J. Climatol., 26, 15671585.

    • Search Google Scholar
    • Export Citation
  • Stockdale, T. N., , Anderson D. L. T. , , Alves J. O. S. , , and Balmaseda M. A. , 1998: Global seasonal rainfall forecasts using a coupled ocean–atmosphere model. Nature, 392, 370373.

    • Search Google Scholar
    • Export Citation
  • Tennant, W. J., , and Hewitson B. , 2002: Intra-seasonal rainfall characteristics and their importance to the seasonal prediction problem. Int. J. Climatol., 22, 10331048.

    • Search Google Scholar
    • Export Citation
  • Tippett, M. K., , Barnston A. G. , , and DeWitt D. , 2005: Statistical correction of tropical Pacific sea surface temperature forecasts. J. Climate, 18, 51415162.

    • Search Google Scholar
    • Export Citation
  • Troccoli, A., , Harrison M. , , Anderson D. L. T. , , and Mason S. J. , 2008: Seasonal Climate: Forecasting and Managing Risk. NATO Science Series on Earth and Environmental Sciences, Vol. 82, Springer, 467 pp.

    • Search Google Scholar
    • Export Citation
  • Van den Dool, H. M., 1994: Searching for analogues, how long must we wait? Tellus, 46A, 314324.

  • Van den Dool, H. M., , and Toth Z. , 1991: Why do forecasts for “near normal” often fail? Wea. Forecasting, 6, 7685.

  • Van Heerden, J., , Terblanche D. E. , , and Schulze G. C. , 1988: The Southern Oscillation and South African summer rainfall. Int. J. Climatol., 8, 577597.

    • Search Google Scholar
    • Export Citation
  • Van Rooy, M. P., 1972: Climate of South Africa: District rainfall for South Africa and the annual march of rainfall over southern Africa, part 10. South African Weather Bureau Tech. Note WB 35, Dept. of Transport, Pretoria, South Africa, 116 pp.

  • Washington, R., , and Preston A. , 2006: Extreme wet years over southern Africa: Role of the Indian Ocean sea surface temperatures. J. Geophys. Res., 111, D15104, doi:10.1029/2005JD006724.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. 2nd ed. Academic Press, 627 pp.

  • Yeh, S.-W., , Wu R. , , and Kirtman B. P. , 2007: Impact of the Indian Ocean on ENSO variability in a hybrid coupled model. Quart. J. Roy. Meteor. Soc., 133, 445457.

    • Search Google Scholar
    • Export Citation
Save