1. Introduction
Seasonal forecasts of precipitation and further meteorological variables are needed for the early warning of droughts and floods in many regions of the world. This information is useful for decision-making to mitigate the negative impacts of hydrometeorological extremes in agriculture, water resources management and further sectors. In many regions of the world, seasonal forecasts are provided by regional climate outlook forums (RCOF; Ogallo et al. 2008; Semazzi 2011), consortia of climate experts from national, regional, and international institutions with the objective of producing seasonal climate outlooks once or several times a year for a geographical region of interest and to disseminate this information to stakeholders and other users. Nowadays, 19 RCOF have been established across the world (WMO 2017). Although some RCOF have provided outlooks for almost two decades (Ogallo et al. 2008), evaluation studies of RCOF are still very limited (Hansen et al. 2011). An example is presented by Mason and Chidzambwa (2009) for the African RCOF, but no detailed description of the forecast methods used by the African RCOF is given. Another study was shown by Mwangi et al. (2014), who investigated the performance of the East Africa RCOF for several rainfall stations in Kenya, Uganda, and Tanzania in comparison with the coupled general circulation model (CGCM) of the European Centre of Medium-Range Weather Forecasts (ECMWF) used for seasonal forecasting (ECMWF-S4; Molteni et al. 2011), based on a qualitative analysis.
One of the earliest RCOF products is the precipitation outlook provided by the West African RCOF (Ogallo et al. 2008). It is one of the most important hydrometeorological services delivered by the weather services in West Africa, which was established in 1998 (Tarhule and Lamb 2003; Hansen et al. 2011). It is a forecasting initiative of the national weather and hydrological services of West African and some central African countries, coordinated by the African Centre of Meteorological Applications for Development (ACMAD) and the Centre Regional de Formation et d’Application en Agrométéorologie et Hydrologie Opérationelle (AGRHYMET Regional Centre). The forum is usually complemented by experts from several Global Producing Centres for Long-Range Forecasts and other climate specialists. Until 2013, the West African RCOF was known under its French name, Prévisions Saisonnières en Afrique de l’Ouest (PRESAO; Mason and Chidzambwa 2009). In 2014, this initiative was split into Prévisions Saisonnières Agro-Hydro-Climatiques pour la Zone Soudano-Sahélienne (PRESASS; ACMAD 2017a) and Prévisions Saisonnières Agro-Hydro-Climatiques pour la Zone du Golf de Guinée (PRESAGG; ACMAD 2017b) to account for the differences in precipitation regime between the Gulf of Guinea and the Sudan–Sahel regions of West Africa. These new products of the West African RCOF provide seasonal precipitation forecasts for the early monsoon period (March–May) for the countries of the Gulf of Guinea and for the monsoon peak (July–September) for the entire region of West Africa (ACMAD 2017b).
A first assessment of the seasonal precipitation forecast of the West African RCOF was performed by Mason and Chidzambwa (2009) for the period from 1998 to 2007 in comparison with grid-based observations. They indicated that the seasonal precipitation forecast for the lower (below normal) and the upper third (above normal) of the climatological distribution have some positive skill but also displayed several limitations such as lack of sharpness. Tall et al. (2012) and Braman et al. (2013) highlighted how the PRESAO forecast was used by the Red Cross to improve flood preparedness in several West African countries in 2008.
The development of operational seasonal forecasting models that complement the West African RCOF is relatively rare for West Africa, although many advances have been made in the field of downscaling of meteorological variables for this region. An overview is given by Paeth et al. (2011). However, many downscaling studies focused on the development of regional climate models and the evaluation of their performance (e.g., Nikulin et al. 2012; Sylla et al. 2013; Arnault et al. 2016; Klein et al. 2015), the provision and analysis of regional climate projections (e.g., Laprise et al. 2013; Mariotti et al. 2014), and the investigations of regional land–atmospheric interactions (e.g., Klein et al. 2017). According to Philippon et al. (2010) seasonal forecast applications for West Africa can be distinguished in purely statistical approaches (e.g., Garric et al. 2002), CGCMs and statistical downscaling approaches based on model output statistics. The actual skill of the West African monsoon rainfall of CGCMs was explored in detail by, for example, Philippon et al. (2010) and Rodrigues et al. (2014) for seasonal forecasting. First statistical downscaling approaches were shown, for example, by Paeth and Hense (2003) and Bouali et al. (2008). Ndiaye et al. (2009, 2011) applied model output statistics on regional wind patterns simulated by CGCMs to improve the prediction of seasonal precipitation amounts for the Sahel. Batté and Déqué (2011) used a quantile–quantile transformation for bias correction of precipitation from several CGCMs for West Africa and other regions in Africa. Manzanas (2017) used the analog method for downscaling precipitation and temperature for Senegal and clearly showed the added valued of the statistical approach in comparison with raw output from several GCMs. A unique dynamical downscaling approach is illustrated by Siegmund et al. (2015), who downscaled real-time forecasts from the Climate Forecast System, version 2 (CFS2; Saha et al. 2014), to provide high-resolution ensemble forecasts for several West African countries for the rainy season in 2013. An operational system for hydrological drought monitoring and forecast for Africa based on CFS2 forecasts was also presented by Yuan et al. (2013) and Sheffield et al. (2014).
This study gives an overview of the information and methods used by the West African RCOF to provide seasonal precipitation outlooks for their region of interest. This information was not provided with such detail in previous studies, but is necessary to better understand and further develop these RCOF products. The study is complemented by a detailed verification of the West African RCOF product PRESAO, which was issued from 1998 to 2013 for the monsoon peak. The seasonal precipitation outlooks are specifically evaluated across West Africa on different spatial scales using a novel precipitation database of rain gauge measurements along with a standard gridded observational dataset. Several verification methods ranging from scalar measures to diagnostic diagrams are used to determine six attributes of forecast quality: bias, accuracy, resolution, reliability, sharpness, and skill. Here, we follow the verification studies presented by Wilks and Godfrey (2002) and Mason and Chidzambwa (2009) for seasonal precipitation products. In addition to these studies, the RCOF forecasts are also evaluated with respect to a binary warning system and combined with a cost–loss model to determine the economic value of seasonal precipitation forecasts for the early warning of wet and dry seasons on a theoretical basis. This is a crucial step in forecast verification to better highlight the usefulness of forecasting products. It also enables the estimation of the forecast value, which is another type of forecast goodness beside forecast quality (Murphy 1997). Since the verification of RCOF products can be very uncertain because of the small sample size available for evaluation (i.e., 16 years in this study), a bootstrap algorithm is presented to add an additional uncertainty analysis to the verification process.
2. Description of the West African regional climate outlook forum
A basic overview of the forecasting procedure used by the West African RCOF is illustrated in Fig. 1. In the first step, the national weather services of West Africa perform countrywide seasonal precipitation forecasts using statistical techniques. The statistical methods are based on objective (automated) approaches complemented by subjective (manual) techniques (e.g., Wilks 2011, his sections 7.5 and 7.8; Bárdossy et al. 2002; Yarnal 1992, p. 10). The objective techniques are based on statistical downscaling techniques (e.g., Maraun et al. 2010) to improve the precipitation forecasts from CGCMs used for seasonal forecasting like CFS2. The subjective approaches are done through classical statistical forecasting using observations and introducing a time lag for the prediction (Wilks 2011, p. 255) by utilizing the experiences of the meteorologists.
Conceptual diagram of the forecasting procedure used by West African RCOF (P = precipitation, U = u-wind component, AM = analog method, CC = canonical correlation, MLR = multiple linear regression, PC = principal component analysis, TAO = tropical Atlantic Ocean, TP = tropical Pacific, IO = Indian Ocean, NMS = national meteorological services, GPC = Global Producing Centres of Long-Range Forecasts, and MMF = multimodel forecast; other acronyms are defined in the text).
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
Conceptual diagram of the forecasting procedure used by West African RCOF (P = precipitation, U = u-wind component, AM = analog method, CC = canonical correlation, MLR = multiple linear regression, PC = principal component analysis, TAO = tropical Atlantic Ocean, TP = tropical Pacific, IO = Indian Ocean, NMS = national meteorological services, GPC = Global Producing Centres of Long-Range Forecasts, and MMF = multimodel forecast; other acronyms are defined in the text).
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
Conceptual diagram of the forecasting procedure used by West African RCOF (P = precipitation, U = u-wind component, AM = analog method, CC = canonical correlation, MLR = multiple linear regression, PC = principal component analysis, TAO = tropical Atlantic Ocean, TP = tropical Pacific, IO = Indian Ocean, NMS = national meteorological services, GPC = Global Producing Centres of Long-Range Forecasts, and MMF = multimodel forecast; other acronyms are defined in the text).
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
An example of subjective forecasting is the approach used by the national weather service of Mauritania, based on an analog method. This approach is one of the simplest and intuitive statistical methods with a long history in weather forecasting (e.g., Lorenz 1969) and climate downscaling (Zorita and von Storch 1999), but the analog method is usually used objectively (e.g., Obled et al. 2002; Marty et al. 2012; Horton et al. 2017). Specific seasonal forecasting applications are demonstrated by, for example, Mullan and Thompson (2006) for temperature, and Charles et al. (2013) for precipitation. The methodology used by the Mauritanian weather service is based on a visual comparison of the current sea surface temperature (SST) conditions of the Atlantic Ocean close to the Mauritanian coastline with SST conditions of past years to identify the most similar or closest SST situations (analog). The seasonal precipitation amount for July–September (JAS) in the analog year is then used as information to forecast the seasonal precipitation amount for JAS of the target year. The visual comparison of the SST maps and the identification of the analog are done by meteorologists. Therefore, the performance of the approach strongly depends on the experiences and the specific rules followed by the experts.
An example of an objective approach is the forecasting model applied by the National Meteorological Agency of Burkina Faso (ANAM-BF). Their approach is based on the Climate Predictability Tool (CPT; Mason 2011; Mason and Tippett 2017), which uses multiple regression and principal components analysis. These approaches are also widely used in weather forecasting and climate downscaling (e.g., Wilby and Wigley 1997; Wilks 2011, p. 215). Specific applications for seasonal precipitation forecasting are demonstrated by Folland et al. (1991), Shabbar and Barnston (1996), and Rajeevan et al. (2007). In the case of ANAM-BF, the predictand is the seasonal precipitation for several forecast periods (e.g., July–August, JAS, or August–October). The predictors are mainly SST and atmospheric variables such as 850-hPa winds, surface temperature, and precipitation from the North American Multi-Model Ensemble (NMME; National Oceanic and Atmospheric Administration 2017; Kirtman et al. 2014).
Further sources of information for formulating the RCOF outlooks are the latest observed SST anomalies and expected trends over the tropical Atlantic Ocean, the Indian Ocean, the Pacific Ocean, and the Mediterranean Sea (e.g., ACMAD 2016). This information is complemented by global SST and precipitation forecasts provided by the Global Producing Centres for Long-Range Forecasts (Graham et al. 2011). In the past, global seasonal forecasts from the CFS of NCEP (Saha et al. 2006, 2014), the global model of the ECMWF (Stockdale et al. 2011; ECMWF 2017), the Global Seasonal Forecast System (GloSea) of the Met Office (Vellinga et al. 2013; MacLachlan et al. 2015), and the ARPEGE model of Météo France (Déqué et al. 1994) were used. Additionally, information from the multimodel approach of the U.S. International Research Institute for Climate and Society (IRI) was applied (Barnston et al. 2010; Kirtman et al. 2014).
The second step of the West African RCOF is the harmonization process, where the different sources of information are merged to provide a consensus outlook for West Africa. The base of the harmonization is a subjective reinterpretation of the countrywide precipitation forecasts using forecasts from the global models and the outcomes of the observed SST analysis. The harmonization is done by all participating national weather services and other experts during the annual meetings of the West African RCOF. Based on their agreement, several precipitation zones are delineated for regions where precipitation anomalies are expected. Additionally, forecast probabilities are assigned to the categories above normal, near normal, and below normal for each zone.
A typical output of an RCOF outlook is shown in Fig. 2 for the year 2013. Average conditions for the reference periods (climatological average) are indicated by uniformly distributed forecast probabilities of 1/3. Right-skewed distributed forecast probabilities such as for zone I in Fig. 2 indicate regions that are expected to be wetter in comparison to the reference period (i.e., 1971 to 2000), whereas left-skewed distributed forecast probabilities for zone II and III of Fig. 2 show areas that deviate from the climatological average toward drier conditions. Any white areas illustrate regions where the forecasts are identical to the climatological average or where no forecasts were issued (usually north of the Sahel region).
An example of the West African RCOF precipitation outlook for JAS 2013. Seasonal precipitation forecasts are visualized as forecast probabilities for the tercile-based precipitation categories above normal, near normal, and below normal (top to bottom) for three different zones. Zone I is expected to be wetter in comparison with the reference period, whereas zones II and III are expected to be drier. The map was drawn from the original forecast map, published in the seasonal climate bulletin of ACMAD.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
An example of the West African RCOF precipitation outlook for JAS 2013. Seasonal precipitation forecasts are visualized as forecast probabilities for the tercile-based precipitation categories above normal, near normal, and below normal (top to bottom) for three different zones. Zone I is expected to be wetter in comparison with the reference period, whereas zones II and III are expected to be drier. The map was drawn from the original forecast map, published in the seasonal climate bulletin of ACMAD.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
An example of the West African RCOF precipitation outlook for JAS 2013. Seasonal precipitation forecasts are visualized as forecast probabilities for the tercile-based precipitation categories above normal, near normal, and below normal (top to bottom) for three different zones. Zone I is expected to be wetter in comparison with the reference period, whereas zones II and III are expected to be drier. The map was drawn from the original forecast map, published in the seasonal climate bulletin of ACMAD.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
The last step of the forecasting procedure is the communication and dissemination of the precipitation forecasts to users in agriculture, water resources management, disaster management agencies and other sectors. This is usually done during a broadcast press conference directly after the meeting of the meteorological experts called the forum. During the forum, the RCOF outlook is presented and related advices are given to decision- and policy makers. The outcomes are further published via bulletins, mailing lists, radio broadcast, seminars, and websites of ACMAD and AGRHYMET Regional Centers.
The characteristics of the PRESAO outlooks used for verification in this study are listed in Table 1. The information was collected from the pure forecast maps (1998–2003) and seasonal climate bulletins (2004–13) provided by ACMAD. The forecasts are usually issued for the West African countries including Chad and Cameroon. Although for some years, the forecasts were also provided for other countries like Sudan or the Central African Republic. The forecast period is JAS. The number of precipitation zones deviating from the climatological average ranges between 3 and 5. Table 1 also indicates that the RCOF meetings usually took place in May. Hence, the lead time of the precipitation forecasts is slightly more than one month. For four years (2000, 2001, 2005, and 2008), the original forecast maps were not available, and updated products were used for verification. The updated forecasts are usually produced by ACMAD/AGRHYMET in the end of June but without involvement of the entire RCOF community. Thus, the lead time of these updated forecast products is shorter, and this may impact forecast performance.
Basic characteristics of the West African RCOF precipitation outlook for JAS used in this study for an evaluation—WA = West African countries, CD = Chad, and CA = Cameroon; PZ is the number of precipitation zones; date is the date of issue; NHMS and GPC are the number of National Hydrological Meteorological Services and Global Producing Centres for Long-Range Forecasts, respectively, who participated at the meeting (o indicates that GPC or NHMS were part of the initiative but there is no information about the number of institutions; an em dash means there is no information); source = the source of information (M = map; B = bulletin).
3. Study region and observational datasets
The verification of the RCOF outlook is done for four climatological zones (Sahel, Sudan–Sahel, Sudan, and Guinean zone) and five major river basins (Volta, Senegal, and three Niger subbasins; Fig. 3). The climatological zones are located in a rectangular domain ranging from the Sahel to the Guinean coastline (18°–4°N) and from central Côte d’Ivoire to east Nigeria (5°W–5°E). The main reason for using this domain is the availability of a novel precipitation database based on rain gauge measurements (section 3a), complementing the Global Precipitation Climatology Centre (GPCC) reanalysis dataset (section 3b). In comparison to many further studies we divided this domain in four climatological zones instead of three or two (e.g., Omotosho and Abiodun 2007; Panitz et al. 2014; Dieng et al. 2017) to better reflect the strong differences of the precipitation climatology within this region (section 3c and Salack et al. 2018b).
Climatological zones (Sahel, Sudan–Sahel, Sudan, and Guinean) and river basins (Volta, Senegal, upper Niger, central Niger, and Sokoto) used for an evaluation of the PRESAO forecasts. In addition, the locations of the precipitation stations of the WASCAL database are indicated as black dots.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
Climatological zones (Sahel, Sudan–Sahel, Sudan, and Guinean) and river basins (Volta, Senegal, upper Niger, central Niger, and Sokoto) used for an evaluation of the PRESAO forecasts. In addition, the locations of the precipitation stations of the WASCAL database are indicated as black dots.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
Climatological zones (Sahel, Sudan–Sahel, Sudan, and Guinean) and river basins (Volta, Senegal, upper Niger, central Niger, and Sokoto) used for an evaluation of the PRESAO forecasts. In addition, the locations of the precipitation stations of the WASCAL database are indicated as black dots.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
a. WASCAL precipitation database
The West African Science Service Centre on Climate Change and Adapted Land Use (WASCAL; www.wascal.org) precipitation database has recently been established alongside the establishment of WASCAL with a specific focus on Burkina Faso, Ghana, and Benin. A preliminary version of these data was already used for the evaluation of a regional climate model (Dieng et al. 2017), the analysis of the Ouagadougou storm event in 2009 (Engel et al. 2017) and partly used for the identification of thresholds defining extreme rainfall events (Salack et al. 2018a,b). The database used for verification consists of 118 precipitation stations shown in Fig. 3. It was provided by the meteorological services of Burkina Faso, Benin, and Ghana for a period ranging from 1960 to 2010. In addition, daily measurements from the Global Historical Climatology Network (Menne et al. 2012), the African Monsoon Multidisciplinary Analysis (AMMA) database (Fleury et al. 2011) and two databases from the Global Change and the Hydrological Cycle (GLOWA)-Volta project are used (van de Giesen et al. 2002). The quality control of the individual precipitation subsets is based on standard quality controls similar to approaches used at the Global Historical Climatology Network (GHCN; Durre et al. 2010; Menne et al. 2012) that are complemented by geostatistical approaches (e.g., spatial correlogram) for quality control and merging.
b. GPCC reanalysis dataset
In addition to the point measurements, gridded precipitation data from the GPCC reanalysis, version 7 (Becker et al. 2013), are used as information for comparison. This is a global precipitation dataset, which is often used for a climate model evaluation in West Africa (e.g., Diallo et al. 2012; Nikulin et al. 2012; Klein et al. 2015). The GPCC reanalysis version is available from 1900 to 2013 in three spatial resolutions (0.5° × 0.5°, 1.0° × 1.0°, and 2.5° × 2.5°). In this study, the reanalysis version with the intermediate resolution is selected. This information was used to calculate the seasonal precipitation amounts for JAS for each grid point and the corresponding areal averages for the target regions.
The GPCC and WASCAL datasets are not completely independent because GPCC also contains measurements of in situ stations that are part of the WASCAL database. The main advantage of the WASCAL database is that evaluation on the point scale is possible. It is noted that there is no better alternative to GPCC and the WASCAL database because merged satellite-gauge precipitation products do not completely cover the reference periods and other station-based observational data products usually rely on less dense measurement networks than GPCC, as shown by Lorenz and Kunstmann (2012) for Africa and other regions of the world.
c. Seasonal and decadal precipitation variability
The climatological zones used for verification are characterized by a strong north-south precipitation gradient (283 mm yr−1 in the Sahel to 1428 mm yr−1 in the Guinean zone; Table 2) as a result of a longer rainy season for the equatorward zones (Fig. 4). An important seasonal characteristic for the target season of the West African RCOF is the “little dry season” along the coastline from Côte d’Ivoire to west Nigeria, leading to a bimodal monthly precipitation regime for the Guinean zone with a local minimum in August. It is a specific regional West African monsoon (WAM) feature modulating the general WAM picture for this latitude. All river basins are characterized by a unimodal precipitation regime (not shown) with a peak value in August, but the precipitation regime of the equatorward basins (the Volta and the upper Niger basin) is much wetter in comparison with the other watersheds. Another important characteristic is the distinct decadal variability (e.g., Nicholson et al. 2018) of the seasonal precipitation amount for the Sahel and the Sudan–Sahel zones (Fig. 4), with much drier conditions during the 1970s and 1980s (Masih et al. 2014) and changing rainfall characteristics (e.g., Salack et al. 2016). This decadal variability needs to be considered by the RCOF experts and makes the formulation of unbiased seasonal precipitation forecasts particularly challenging for this region.
Some basic characteristics for the target regions: A = domain size (104 km2); Pa = mean annual precipitation amount (1961–90; mm); Ps = areal seasonal precipitation amount for JAS (1961–90; mm); LT1 and UT1 are the lower and upper terciles, respectively, of Ps (mm) for JAS (1961–90); LT2 and UT2 are the lower and upper terciles, respectively, of Ps (mm) for JAS (1971–2000); Pd and Pw are the precipitation thresholds (mm) for dry and wet years, respectively (1998–2013); GPCC reanalysis, version 7.
(a) Monthly precipitation regimes for the climatological zones used in this study, and (b) decadal variability of standardized seasonal precipitation amount for JAS for the Sahel and Sudan–Sahel zone using a z-score transformation. In addition, the PRESAO reference periods and outlook (verification) period are indicated. Both diagrams are based on GPCC reanalysis, version 7, for 1950–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
(a) Monthly precipitation regimes for the climatological zones used in this study, and (b) decadal variability of standardized seasonal precipitation amount for JAS for the Sahel and Sudan–Sahel zone using a z-score transformation. In addition, the PRESAO reference periods and outlook (verification) period are indicated. Both diagrams are based on GPCC reanalysis, version 7, for 1950–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
(a) Monthly precipitation regimes for the climatological zones used in this study, and (b) decadal variability of standardized seasonal precipitation amount for JAS for the Sahel and Sudan–Sahel zone using a z-score transformation. In addition, the PRESAO reference periods and outlook (verification) period are indicated. Both diagrams are based on GPCC reanalysis, version 7, for 1950–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
4. Verification measures and bootstrap algorithm
In this section, the verification scores used in this study to measure forecast quality and value are briefly described along with the bootstrap algorithm. For more information regarding the verification scores, we refer to Wilks (2011, p. 301) and Jolliffe and Stephenson (2012).
a. Forecast quality
In the case of the West African RCOF outlooks, the allowable forecast probabilities ranged between 0.05 to 0.5, with an increment of 0.05. In addition, ⅓ was used for those regions, where the forecast was identical to climatology. The weight gk is equal to the relative frequency of a forecast probability, that is, how often a discrete value of an allowable forecast probability was issued by the RCOF experts in relative terms. If the reliability is zero, the conditional observed frequencies are equal to the forecast probabilities and the forecasts of the RCOF outlooks are therefore considered perfectly reliable (no conditional bias).
It describes the ability of the RCOF outlooks to separate between precipitation events with different frequencies of occurrence. The larger the differences between the conditional mean and the unconditional mean, the better the resolution of the RCOF outlooks. The resolution of RCOF outlook tends to zero when the subsample means are close to sample climatology.
This measure linearly relates the BS of the RCOF outlooks to the BS of a climatological forecast BSc (based on the sample climatology) and the score of a perfect forecast BSp. Like many other skill scores, the BSS ranges between −∞ and 1. A gain in forecast accuracy of the RCOF outlooks compared to a climatological forecast is indicated by a score greater than 0. In addition, normalized measures of reliability BRL and resolution BRS are calculated (Toth et al. 2003; Wilks 2011, p. 333) where 0 for BRL and BRS indicates RCOF outlooks with perfect resolution and reliability.
The precipitation thresholds used for the quality assessment in section 5 are the terciles from two different reference periods (1961–90 and 1971–2000). Since the exact year of the change of the reference periods is unknown, we assume on the basis of a linear interpolation that the new reference period was selected in 2010 by the RCOF community. Before 2003, we use 1961–90, which is in line with Mason and Chidzambwa (2009). It is noted that the terciles between both periods are relatively similar (Table 2) so that any lack of information regarding the exact year of this change has only minor impacts on the quality measures (not shown). Since the verification period is wetter in comparison with both reference periods (Fig. 4b), below-normal situations were relatively seldom for several target regions. The outcomes presented in section 6 is therefore based on precipitation thresholds computed from the data of the verification period to divide the sample exactly into 5 wet years, 5 dry years, and 6 usual years.
b. Forecast value
Seasonal forecasts are used by stakeholders for planning in their respective socioeconomic sectors. It is therefore important to determine the value of seasonal forecasts for decision-making situations (e.g., no drought, drought) to illustrate the potential benefit for users. This task is done in forecast verification using cost–loss approaches that combine the outcomes of a binary warning system (hit, false alarms, miss, and correct rejection) with a cost–loss model (e.g., Katz and Murphy 1997; Wilks 2011, p. 377; Richardson 2012). Specific applications of these approaches are illustrated by, for example, Richardson (2001) for heavy rainfall, Roulin (2007) for flood forecasting, and Batté and Déqué (2011) for seasonal forecasts of monthly precipitation amounts.
In this study, a static cost–loss approach is used. It assumes that the decision-maker is risk neutral and that decisions are only influenced by the forecast. A decision-maker can decide between “no action” and “protective action.” If the decision-maker issues an alarm, protective action is taken, generating costs C for an alarm but preventing the loss L. If the decision-maker issues no alarm, there are no alarm costs, but losses occur in the case of a miss. The values of C and L are user dependent, and the cost–loss ratio α = C/L is therefore a measure defining a specific user.
This measure is called the economic value V, which can be used to determine the potential forecast value of an early warning system. It is a function of the hit rate H, the false-alarm rate F, the climatological frequency of the event s, and the cost–loss ratio α. It ranges between −∞ and 1. A positive value indicates a warning system that produces lower expenses in comparison with the reference warning system. The references are the optimum expenses of two naive warning systems producing “permanent warnings” or “no warnings.” The economic value also depends strongly on the probability threshold pt used for decision-making (section 6a), which is not necessarily identical to the climatological frequency of the event to be verified. Decision thresholds are used in warning situations, to convert probability forecasts to binary values (Richardson 2012). If a forecast probability for below normal and above normal is larger than this threshold, a warning is given, and otherwise there is no warning. Any variation of the probability threshold allows an increase or decrease of the number of warnings.
The economic value is maximized when the cost–loss ratio is equal to the occurrence frequency. In this specific case, Eq. (5) can be simplified to V = H − F (Richardson 2012). The maximum economic value Vmax can be therefore directly estimated by determining the difference between the hit rate and the false-alarm rate. Note that Vmax is also equal to the Peirce skill score. This skill score is characterized by a number of desirable properties such as true equitability and difficulty to hedge that are needed for a reliable verification (Hogan and Mason 2012, p. 46).
c. Estimation of the confidence intervals using bootstrap
The verification of seasonal forecasts is often uncertain because of the limited sample of verification data. To determine this specific uncertainty, we apply a bootstrap approach (Efron and Tibshirani 1994). The estimation of confidence intervals for verification measures is often based on parametric approaches (e.g., Hogan and Mason 2012; Bliefernicht and Bárdossy 2007). The main advantage of bootstrap approaches is that no assumptions must be made regarding the distribution of the verification data to better account for data asymmetries. The bootstrap approach used in our study is based on the approaches for pairwise data shown by Efron and Gong (1983) for the estimation of confidence intervals of the Pearson correlation. Instead of using the correlation, however, we calculated the confidence intervals for Vmax for our n forecast–observation pairs based on H and F. The algorithm consists of six steps:
Definition of the event threshold xt, probability threshold pt, and bootstrap sample size m.
Random selection with replacement of n forecast–observation pairs from the time series of observed precipitation amounts x = (x1, x2, …, xn) and RCOF forecast probabilities f = (f1, f2, …, fn).
Transformation of the observations and forecasts into binary values using xt and pt.
Calculation of H, F, and Vmax.
Repetition of steps 2–4 m times to determine m realizations of Vmax.
Calculation of the 10% and 90% quantiles on the basis of the m realizations of Vmax.
5. Results and discussion
a. Forecast quality of RCOF probability forecasts
The reliability diagrams of the RCOF forecasts for the seasonal precipitation amounts at the precipitation stations in Burkina Faso, Ghana, and Benin are shown in Fig. 5. Perfectly reliable forecasts are indicated by data pairs that follow the 1:1 line of the diagram, and the reliability measure defined in Eq. (2) would be zero. Further insights about the resolution of the RCOF forecasts are given by this diagram. The higher the deviations between the data pairs (regression line) and the no-resolution line (unconditional observed frequency) are, the higher is the resolution of the RCOF forecast. If the data pairs follow exactly the resolution line, the resolution measure in Eq. (3) is zero.
Reliability diagrams of the West African RCOF forecasts (a) for all categories and for the three precipitation categories (b) above normal, (c) near normal, and (d) below normal for 118 precipitation stations in Burkina Faso, Ghana, and Benin of the WASCAL precipitation database, seasonal precipitation amount (JAS), 1998–2010. In addition, the regression line of the sample is depicted.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
Reliability diagrams of the West African RCOF forecasts (a) for all categories and for the three precipitation categories (b) above normal, (c) near normal, and (d) below normal for 118 precipitation stations in Burkina Faso, Ghana, and Benin of the WASCAL precipitation database, seasonal precipitation amount (JAS), 1998–2010. In addition, the regression line of the sample is depicted.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
Reliability diagrams of the West African RCOF forecasts (a) for all categories and for the three precipitation categories (b) above normal, (c) near normal, and (d) below normal for 118 precipitation stations in Burkina Faso, Ghana, and Benin of the WASCAL precipitation database, seasonal precipitation amount (JAS), 1998–2010. In addition, the regression line of the sample is depicted.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
Figure 5a shows that seasonal precipitation forecasts issued for the three precipitation categories are skillful for the point measurements. They have a moderate to good reliability with tendencies to underestimate particularly higher forecast probabilities. If the verification is done separately for the three precipitation categories, some specific shortcomings of the forecasts can be identified for the individual categories (Figs. 5b–d, Table 3). For instance, the reliability diagram for above normal indicates that the forecast probabilities are too low (BRL = 7%) in comparison with the conditional observed frequency over the entire range of forecast probabilities, with a strong tendency to underforecast the event frequency (bias = 24%). The precipitation regime in West Africa is characterized by a distinct decadal variability and the reference periods were much drier in comparison with the verification period. This effect was probably not foreseen by the RCOF experts. Although the forecasts have some resolution for this category (BRS = 94.6%), the systematic deviations reduce the accuracy of the forecasts, leading to a BSS of −1.6%. However, a negative BSS does not automatically indicate a nonvaluable forecast, as illustrated by, for example, Richardson (2012). The seasonal precipitation forecasts for below normal are more accurate. They have similar resolutions (BRS = 94.8%) in comparison with above normal, but a better reliability (BRL = 1.5%), especially for lower forecast probabilities, resulting in a positive skill of 3.7%. The forecasts for near normal are severely limited, because they are not able to resolve the event The lack of skill for this category is a common limitation of tercile-based seasonal forecasts products (see Wilks 2013; van den Dool and Toth 1991) and is not only related to subjective seasonal forecasts products. Moreover, the event probability of this category is also overforecast by roughly 29%. Forecasters tend to use higher forecast probability for near normal to avoid the risk of producing a wrong forecast for the other two categories, which was also shown by Mason and Chidzambwa (2009) for the other African RCOF precipitation outlooks. This seems to be a general shortcoming of seasonal outlooks formulated subjectively like RCOF products.
The performance of the RCOF outlooks for different reference datasets (RD) and scales: A = point observations of the WASCAL database (1998–2010); B = gridded measurements of the GPCC reanalysis, version 7, for four climatological zones (1998–2010); C = as for B, but for an extended verification period (1998–2013); D = areal averages of the GPCC measurements of the four climatological zones (1998–2013). Cat = precipitation category; n = sample size; 
Relatively similar results are obtained in comparison with gridded GPCC measurements for the four climatological zones over the same and a slightly extended verification period (1998–2013; Fig. 6 and Table 3). The near-normal category is characterized by a lack of skill and an overforecasting, whereas the forecasts for the other two categories are skillful but the resolution is slightly lower and strong unconditional biases for above normal is not present anymore. The striking differences, such as the unconditional bias between both datasets (GPCC and WASCAL), are mostly caused by the reference datasets and not by the extended verification period, as shown in Table 3. The verification scores, computed for both periods (1998–2010 vs 1998–2013) using GPCC, differ only slightly from each other (cf. B with C). When the spatial scale is aggregated from the GPCC pixels to an areal average for the four climatological zones, the quality of the RCOF outlooks slightly increases, leading to positive BSS for below normal and above normal. This improvement can be somewhat expected because the predictability of precipitation is much lower at the grid or point scale as a result of high local precipitation variability in West Africa.
Reliability diagrams of the West African RCOF outlooks (a) for all categories and for the three precipitation categories (b) above normal, (c) near normal, and (d) below normal, GPCC reanalysis, version 7, for climatological zones Sahel, Sudan–Sahel, Sudan, and Guinean, seasonal precipitation amount (JAS), 1998–2013. In addition, the regression line of the sample is depicted.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
Reliability diagrams of the West African RCOF outlooks (a) for all categories and for the three precipitation categories (b) above normal, (c) near normal, and (d) below normal, GPCC reanalysis, version 7, for climatological zones Sahel, Sudan–Sahel, Sudan, and Guinean, seasonal precipitation amount (JAS), 1998–2013. In addition, the regression line of the sample is depicted.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
Reliability diagrams of the West African RCOF outlooks (a) for all categories and for the three precipitation categories (b) above normal, (c) near normal, and (d) below normal, GPCC reanalysis, version 7, for climatological zones Sahel, Sudan–Sahel, Sudan, and Guinean, seasonal precipitation amount (JAS), 1998–2013. In addition, the regression line of the sample is depicted.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
An important aspect for the interpretation of the quality of RCOF forecasts is the underlying distribution of the forecasts’ probabilities, that is, how often a probability value was issued by the experts. This information is usually given as a histogram within a reliability diagram (e.g., Wilks 2011, p. 338; Wilks and Godfrey 2002). In this study, this information is shown in Fig. 7 for the samples used in Fig. 5 to better highlight the differences between the precipitation categories. The histograms for the reliability diagrams in Fig. 6 have very similar distributions and are therefore not shown here. The histograms of the probability forecasts can also be interpreted as a type of sharpness diagram for probability forecasts (Murphy 1997). Sharpness is another important attribute of forecast quality, like reliability and resolution (e.g., Wilks 2011, p. 305; Katz and Murphy 1997), but, unlike the other attributes, it is independent from any observations and is therefore only based on pure forecasts.
Histograms for the forecast probabilities (0.1, 0.15, 0.2, 0.25, 0.3, ⅓, 0.35, 0.4, 0.45, 0.5, 0.55, and 0.6) issued by the West African RCOF for (top left) the entire sample and the three precipitation categories (top right) above normal, (bottom left) near normal, and (bottom right) below normal for 118 precipitation stations in Burkina Faso, Ghana, and Benin of the WASCAL database, seasonal precipitation amount (JAS), 1998–2010.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
Histograms for the forecast probabilities (0.1, 0.15, 0.2, 0.25, 0.3, ⅓, 0.35, 0.4, 0.45, 0.5, 0.55, and 0.6) issued by the West African RCOF for (top left) the entire sample and the three precipitation categories (top right) above normal, (bottom left) near normal, and (bottom right) below normal for 118 precipitation stations in Burkina Faso, Ghana, and Benin of the WASCAL database, seasonal precipitation amount (JAS), 1998–2010.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
Histograms for the forecast probabilities (0.1, 0.15, 0.2, 0.25, 0.3, ⅓, 0.35, 0.4, 0.45, 0.5, 0.55, and 0.6) issued by the West African RCOF for (top left) the entire sample and the three precipitation categories (top right) above normal, (bottom left) near normal, and (bottom right) below normal for 118 precipitation stations in Burkina Faso, Ghana, and Benin of the WASCAL database, seasonal precipitation amount (JAS), 1998–2010.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
An example of an extremely unsharp forecast is a climatological forecast with probabilities of ⅓ for each of the three precipitation categories and for all forecast situations. In this case, the histogram is characterized by a single bar at the forecast probability ⅓ with a relative frequency of 1 (Fig. 8). The other extreme is a perfectly sharp forecast. The histogram of this forecast only has two bars at the forecast probabilities of 0 and 1, because only 0s and 1s are issued. Based on these baselines, the histograms of Fig. 7 demonstrate that the RCOF forecasts clearly deviate from an unsharp climatological forecast but remain far from being perfectly sharp.
(left) Histograms of a climatological and perfectly sharp probabilistic forecast for a tercile-based category. (right) The histograms of three perfectly reliable theoretical forecasts with a u-shape, uniform, and binomial distribution (skewed distribution).
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
(left) Histograms of a climatological and perfectly sharp probabilistic forecast for a tercile-based category. (right) The histograms of three perfectly reliable theoretical forecasts with a u-shape, uniform, and binomial distribution (skewed distribution).
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
(left) Histograms of a climatological and perfectly sharp probabilistic forecast for a tercile-based category. (right) The histograms of three perfectly reliable theoretical forecasts with a u-shape, uniform, and binomial distribution (skewed distribution).
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
The histograms of the forecast probabilities are also needed for a better interpretation of the quality scores used in this study. The relative frequencies of the histogram are the weights of the resolution and of the reliability term defined in Eqs. (2) and (3). Thus, the shape of the histogram plays a crucial role for the exact value of the quality scores used in this study. This impact is shown for several theoretical forecasts with different histogram shapes and widths (Table 4). The histogram with the smaller width should mimic the range of forecast probabilities issued by the West African RCOF experts for above normal. The accuracy of a perfectly reliable RCOF forecast with forecast probabilities ranging from 0.2 to 0.5 slightly increases in comparison to the imperfectly reliable forecasts, but the skill remains very low. Even in the case of a perfectly sharp forecast with forecast probabilities issued for the same range, the forecasts still have a low resolution and therefore a minor skill (BSS = 10.0%). The same also applies to uniformly distributed forecast probabilities over the entire range. A clear improvement of the skill can only be seen for forecast probabilities issued over the entire range with a uniform or u-shape histogram.
The impact of the underlying distribution of the forecast probabilities (uniform, u-shape, and binomial distributions; Fig. 7) on quality measures as based on several perfect reliable forecasts issued for two forecast probability ranges (0.2–0.5 and 0–1). RCOF-PR = perfect reliable forecasts as based on the distribution of Fig. 6 for above normal. The measures are as in Table 3, with U = uncertainty (%).
To improve RCOF forecasts, one goal could be the maximization of sharpness without losing any reliability. However, sharp forecasts with probabilities tending to 0 or 1 cannot be expected from forecasters, otherwise they would ignore the inherent uncertainty of seasonal precipitation forecasts. This lack of sharpness is a common problem in seasonal forecasting and was also shown for other seasonal precipitation products (Wilks and Godfrey 2002; Barnston and Mason 2011). That is why the resolution of seasonal forecasts is very low, leading to accuracy similar to the accuracy of a climatological forecast, particularly if the forecasts are characterized by additional conditional biases. Thus, the interpretation of the quality of seasonal RCOF forecasts solely based on standard quality scores such as the Brier score may lead to a wrong conclusion (Mason and Chidzambwa 2009; Mason 2012). It is therefore important to consider verification methods such as the reliability and the sharpness diagram to get a better insight of the quality of seasonal forecasts. Moreover, it is highly important to investigate how useful RCOF outlooks are for decision-making, as shown in the following section.
b. Forecast value and use of RCOF precipitation warnings
The economic value of the RCOF precipitation warnings for wet years of the Guinean zone and dry years of the Sudan–Sahel zone is shown in Fig. 9. The users are defined by the cost–loss ratio (see section 4b). For instance, if the losses L of a miss are 10 times as large as the costs C of a false alarm, the user has a cost–loss ratio of C/L = 1/10. The economic value was calculated for several cost–loss ratios (0.025, 0.05, …, 0.975) for a set of probability thresholds [0.12, 0.14, …, 0.48]. The result of this calculation is illustrated in Fig. 9. Each line shows the curve of the economic value for a given probability threshold. The economic values for thresholds of 0.1, 0.2, 0.3, and 0.4 are indicated by contour lines, and the remaining ones are shown by dashed lines. Note that for several decision thresholds the curve of the economic value is identical because of the same underlying binary statistics (H and F).
The economic value V of the RCOF forecasts for dry years of the Guinean zone and wet years of the Sudan–Sahel zone in relation to the cost–loss ratio computed for probability thresholds between 0.1 and 0.5; GPCC reanalysis, version 7, areal seasonal precipitation amount (JAS), 1998–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
The economic value V of the RCOF forecasts for dry years of the Guinean zone and wet years of the Sudan–Sahel zone in relation to the cost–loss ratio computed for probability thresholds between 0.1 and 0.5; GPCC reanalysis, version 7, areal seasonal precipitation amount (JAS), 1998–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
The economic value V of the RCOF forecasts for dry years of the Guinean zone and wet years of the Sudan–Sahel zone in relation to the cost–loss ratio computed for probability thresholds between 0.1 and 0.5; GPCC reanalysis, version 7, areal seasonal precipitation amount (JAS), 1998–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
Figure 9 gives therefore several important insights about usefulness and application of seasonal precipitation forecasts issued by the West African RCOF. It clearly illustrates that valuable precipitation warnings for the prediction of dry years were provided for the Guinean zone for a wide range of users because the economic value is positive for cost–loss ratios between 0.05 and 0.77. Slightly more than 60% of the expenses can be saved in the optimal case for the Guinean zone, since the peak value of the economic value Vmax is 0.62. Similar results are also obtained for the prediction of wet years for the Sudan–Sahel zone. The diagram also shows that any variation of the decision threshold can strongly influence the economic value for a given user and that different probability thresholds must be selected for both examples for obtaining the best economic value.
However, a very simple approach is used for the estimation of the economic value. The approach is based on relative values and does not take into account the real expenses of a warning system as shown in, for example, Katz et al. (1982). The analysis gives insights about the potential forecast value of an early warning system and needs to be extended for determining the forecast value for real-world situations. A further important simplification of the approach is the risk neutrality of users. Thus, repeated false alarms (overwarnings) have no influence on the user’s willingness to respond to future alarms. This problem is discussed in, for example, Barnes et al. (2007) and may also affect the forecast value determined in this study.
The maximum economic value Vmax of the West African RCOF forecast was also calculated for all target regions of this study (Fig. 10). The confidence intervals based on the 10% and 90% quantile are given to illustrate the uncertainty of the estimation of Vmax. Thus, 80% of the bootstrap samples for Vmax range between the 10% and 90% quantiles. Figure 10 indicates that the RCOF outlook can provide valuable precipitation warnings for almost all target regions and event types, with a slightly better performance for dry (average median of Vmax = 0.39) than for wet years (average median of Vmax = 0.34). The median Vmax only tends to zero for the prediction of wet years in the central Niger River basin and dry years for the Sahel zone, indicating that the precipitation warnings based on the RCOF outlook were not better than the naive reference warning systems. The bootstrap analysis shows that there are large uncertainties for the exact estimation of Vmax. This is illustrated by the big differences between both quantiles Q (Q90 minus Q10), which vary between 0.38 and 0.84 for the different target regions. Note that 20% of the bootstrap samples are outside this interval. Thus, the uncertainty will be even larger if more extreme quantiles (e.g., 5% and 95% quantiles) are selected. The diagram also indicates strong regional differences for the forecast values. For instance, the median Vmax of the Guinean zone for the dry and wet seasons is much higher (average median Vmax = 0.54) than that of the Sahel zone (average median Vmax = 0.14). Thus, the precipitation warnings issued by the West African RCOF are much more valuable for the Guinean zone and show only a small benefit for the Sahel region.
The maximum economic value Vmax for the river basins and climatological zones for the five driest and five wettest years. The uncertainty of the estimation of Vmax is also illustrated as based on the 10% and 90% quantiles; GPCC reanalysis, version 7, areal seasonal precipitation amount (JAS), 1998–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
The maximum economic value Vmax for the river basins and climatological zones for the five driest and five wettest years. The uncertainty of the estimation of Vmax is also illustrated as based on the 10% and 90% quantiles; GPCC reanalysis, version 7, areal seasonal precipitation amount (JAS), 1998–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
The maximum economic value Vmax for the river basins and climatological zones for the five driest and five wettest years. The uncertainty of the estimation of Vmax is also illustrated as based on the 10% and 90% quantiles; GPCC reanalysis, version 7, areal seasonal precipitation amount (JAS), 1998–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
The Vmax in Fig. 10 shows the best probability threshold used for precipitation warnings to obtain the maximum economic value. Any variation of this decision threshold may change Vmax. This effect is illustrated in Fig. 11 for wet years of the Sudan–Sahel zone (left panel) and dry years of the Guinean zone (right panel). In the case of the Guinean zone, the highest values of the median Vmax are reached between 30% and 34%. If intuitively the climatological line (event frequency of s = 5/16 = 0.3125) is used as a decision threshold, the median Vmax is therefore high and almost equal to the maximum value. However, this is not the case for the Sudan zone, where the best Vmax is achieved if a much higher decision threshold between 38% and 42% is selected because of the inherent biases of the RCOF outlooks, as illustrated in the previous section.
The maximum economic value Vmax for the (left) five wettest years of the Sudan–Sahel zone and (right) five driest years of the Guinean zone in relation to the probability threshold used for precipitation warnings. The uncertainty margins of the estimation of Vmax are also illustrated as based on the 10% and 90% quantiles; GPCC reanalysis, version 7, areal seasonal precipitation amount (JAS), 1998–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
The maximum economic value Vmax for the (left) five wettest years of the Sudan–Sahel zone and (right) five driest years of the Guinean zone in relation to the probability threshold used for precipitation warnings. The uncertainty margins of the estimation of Vmax are also illustrated as based on the 10% and 90% quantiles; GPCC reanalysis, version 7, areal seasonal precipitation amount (JAS), 1998–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
The maximum economic value Vmax for the (left) five wettest years of the Sudan–Sahel zone and (right) five driest years of the Guinean zone in relation to the probability threshold used for precipitation warnings. The uncertainty margins of the estimation of Vmax are also illustrated as based on the 10% and 90% quantiles; GPCC reanalysis, version 7, areal seasonal precipitation amount (JAS), 1998–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
The best thresholds for the target regions in comparison with climatological line are shown in Fig. 12. In addition, the intervals of the probability thresholds corresponding to a positive value for the median Vmax are illustrated in Fig. 12. The larger this interval is, the more flexibility (or freedom) a decision-maker has in terms of the probability threshold to provide useful warnings for dry and wet seasons. The diagrams therefore highlight several important aspects regarding the use of seasonal precipitation forecasts. First, it clearly shows a difference between the warnings for dry and for wet years. The best probability thresholds for the early warning of dry seasons are below the climatological line. Thus, the warnings for these types of event are more valuable when a probability threshold below the event frequency is selected. The decision-maker must follow a more pessimistic warning strategy, where more warnings (false alarms) are issued, but the number of misses is reduced. However, this is not the case for the warning of wet years in 7 of 9 regions. In this case, a more optimistic warning strategy must be selected, with a probability threshold above the event frequency. Second, the interval illustrates the robustness in selecting a valuable probability threshold for an individual region. In the case of the Guinean zone, this interval is much larger in comparison to other regions like Sokoto and Sahel. The interval ranges between 11% and 44% for the prediction of dry years, and 21% and 44% for wet years for the Guinean zone, whereas the interval tends to zero, for example, for the Sokoto River basin and the Sahel zone for the prediction of dry years.
The interval of the probability threshold where the median of the maximum economic value Vmax is positive for the (left) five wettest and (right) five driest years; GPCC reanalysis, version 7, areal seasonal precipitation amount (JAS), 1998–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
The interval of the probability threshold where the median of the maximum economic value Vmax is positive for the (left) five wettest and (right) five driest years; GPCC reanalysis, version 7, areal seasonal precipitation amount (JAS), 1998–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
The interval of the probability threshold where the median of the maximum economic value Vmax is positive for the (left) five wettest and (right) five driest years; GPCC reanalysis, version 7, areal seasonal precipitation amount (JAS), 1998–2013.
Citation: Journal of Applied Meteorology and Climatology 58, 3; 10.1175/JAMC-D-18-0066.1
6. Summary and conclusions
In this study, we performed a verification of the seasonal precipitation outlooks formulated by the West African RCOF from 1998 to 2013 for the monsoon peak period (JAS). The quality of the RCOF outlooks is different for the tercile-based precipitation categories. The outlooks show some reliability and resolution for below-normal cases and are therefore skillful in comparison to naive climatological forecasts. Similar results are obtained for above normal, but this category is additionally characterized by dry biases. Thus, the verification analysis performed in this study confirmed findings presented by Mason and Chidzambwa (2009) for a shorter period of time.
We also showed that the precipitation forecasts formulated by the West African RCOF for near normal have no skill. The lack of skill for near normal is a typical limitation for tercile-based seasonal forecasts (e.g., van den Dool and Toth 1991; Wilks 2013). However, a detailed analysis of this failure and its potential reasons is rarely investigated. In this study, we showed that precipitation forecasts formulated for near normal are unreliable and show no resolution. Moreover, the forecasts for this category are also characterized by a strong overforecasting. This is very likely due to the so-called forecasters’ risk aversion toward producing wrong forecasts for the other categories (Mason and Chidzambwa 2009) and can therefore affect other RCOF products as well.
In addition to this study, the quality of the RCOF outlooks was also analyzed on different spatial scales. The performance was best for the areal averages of several climatological zones (Sahel, Sudan–Sahel, Sudan, and Guinean) and only decreased slightly for gridded measurements. Our investigation even indicated some skill in comparison to point information. This is a very important finding, as many potential users of the West African RCOF outlook are acting on a local scale like smallholders in rain-fed agriculture.
We also showed that the range of forecast probabilities used by the experts is small because of the inherently high uncertainty of seasonal precipitation forecasts and the forecaster’s aversion to risk. This leads to relatively unsharp forecasts with a low accuracy (or even negative skill), which may lead to the wrong conclusion that the outlook is not useful. The lack of sharpness is a typical problem in seasonal forecasting and was shown for other seasonal precipitation products, too. To overcome this problem, the probabilistic forecasts of the RCOF outlooks were evaluated with respect to a binary warning system and were combined with a cost–loss approach to determine the forecast value on a theoretical basis. This analysis indicated that the West African RCOF outlooks can provide valuable information for the early warning of wet and dry seasons for all target regions, but with strong regional differences.
We also studied the impact of the probability threshold used for decision-making on the forecast value. This is an important analysis giving insight into how probabilistic forecasts of RCOF products can be used for an early warning of extremes to maximize their benefit for specific users. However, any best probability threshold as determined in our analysis may be not robust for future bulletins because of the subjective nature of the RCOF outlooks and when major changes of the forecasting procedure are performed. Our main intention is to illustrate that any variation of the probability threshold can have a strong influence on the forecast value and that an intuitive choice for this threshold (an alarm being given if the forecast probabilities for below normal or above normal exceed the climatological line) is not necessarily the best warning strategy.
The correct interpretation of the forecast probabilities of RCOF outlooks by decision-makers and other users remains an open question. Can they distinguish between a perfect-sharp near-normal forecast (0, 1, 0) and a climatological forecast (⅓, ⅓, ⅓), and does it lead to different actions? Several studies showed that even in the simple case of two-category forecasts, the use of forecast probabilities can lead to misinterpretations (Murphy et al. 1980; Gigerenzer et al. 2005). If users of RCOF outlooks are not able to understand tercile-based precipitation forecasts in a correct way, the forecast value of these products will surely be lower.
A further typical problem of seasonal forecasting studies is the small sample size available for model evaluation, particularly for real-time forecasts such as RCOF products. The outcomes of verification studies in seasonal forecasting are therefore uncertain and conclusions must be considered with caution (Wilks and Godfrey 2002; Mason 2012). This also applies to our study because only 16 RCOF outlooks were used for verification. To better reflect this uncertainty, a bootstrap algorithm for the estimation of the confidence interval of the maximum economic value was presented that can be also used for other verification scores with slight modifications. The bootstrap analysis demonstrated that the estimation of the maximum economic value is characterized by a moderate-to-high uncertainty for the individual target regions, but simultaneously added confidence to the findings of this verification analysis because of the uncertainty margins.
We also described in detail the forecasting procedure used by the regional climate outlook forum in West Africa, one of earliest RCOF products of the world. A relatively complex forecasting procedure is used by the RCOF experts based on various state-of-the-art forecasting approaches and sources of information in combination with expert knowledge. Nevertheless, there are still many ways to complement and improve the current approach. An important issue is the development and provision of additional forecasts information for agricultural and hydrological practices such as the onset of the rainy season (e.g., Moron et al. 2006; Robertson et al. 2009; Vellinga et al. 2013; Bombardi et al. 2017), agrometeorological drought indices (e.g., Dutra et al. 2013; Wetterhall et al. 2015), or hydrological variables (e.g., Andersson et al. 2017). A future task should be also the continuous development of the individual statistical forecasting approaches used by the national meteorological services. Future studies should also consider further development of the harmonization process to provide a consensus precipitation forecasts for West Africa (Mason and Chidzambwa 2009). The implementation of objective procedures for harmonization will probably lack the important knowledge of a forecaster that might be relevant to the forecast situation. However, it will avoid the formulation of risk-averse forecasts and will therefore eliminate crucial shortcomings in the current procedure. It will also increase the transparency of the current forecasting procedure used by the West African RCOF and will therefore facilitate future RCOF development for this region.
Acknowledgments
This work was part of the Core Research Program of the West African Science Service Centre on Climate Change and Adapted Land Use (WASCAL) funded by the Federal Ministry of Education and Research in Germany. We also acknowledge the meteorological services of Burkina Faso, Ghana, and Benin for providing the precipitation data. Special thanks are given to Hanna Arnold who georeferenced and digitalized the PRESAO forecast maps. Author Salack appreciates partial cofunding for this work from the European Research Area for Climate Services (ERA4CS), BMBF through the CIREG project (Grant 690462), and the French Ministry for Foreign Affairs through the APTE-21/FSP-AGRICORA project (MEAE/IRD, AGRICORA axe 1, convention 2016–18). We also want to thank the UPSCALERS project (AURG II-1-074-2016) which is part of the African Union Research Grants financed through the Financing Agreement between the European Commission and the African Union Commission (DCI-PANAF/2015/307-078). We are also grateful to the three anonymous reviewers who provided many valuable comments to improve the final manuscript.
REFERENCES
ACMAD, 2016: Regional climate outlook forum PRESASS-03. Accessed 12 December 2017, http://acmad.net/rcc/atelier/ACMAD_bulletin_PRESASS03.pdf.
ACMAD, 2017a: Regional climate outlook forum for West Africa, Chad and Cameroon. Accessed 6 July 2017, http://acmad.net/rcc/presao.php.
ACMAD, 2017b: Regional climate outlook forum for the Gulf of Guinea countries. Accessed 8 April 2017, http://acmad.net/rcc/presagg.php.
Andersson, J. C., A. Ali, B. Arheimer, D. Gustafsson, and B. Minoungou, 2017: Providing peak river flow statistics and forecasting in the Niger River basin. Phys. Chem. Earth, 100ABC, 3–12, https://doi.org/10.1016/j.pce.2017.02.010.
Arnault, J., S. Wagner, R. Rummler, B. Fersch, J. Bliefernicht, S. Andresen, and H. Kunstmann, 2016: Role of runoff–infiltration partitioning and resolved overland flow on land–atmosphere feedbacks: A case study with the WRF-Hydro coupled modeling system for West Africa. J. Hydrometeor., 17, 1489–1516, https://doi.org/10.1175/JHM-D-15-0089.1.
Bárdossy, A., J. Stehlík, and H. Caspary, 2002: Automated objective classification of daily circulation patterns for precipitation and temperature downscaling based on optimized fuzzy rules. Climate Res., 23, 11–22, https://doi.org/10.3354/cr023011.
Barnes, L. R., E. C. Gruntfest, M. H. Hayden, D. M. Schultz, and C. Benight, 2007: False alarms and close calls: A conceptual model of warning accuracy. Wea. Forecasting, 22, 1140–1147, https://doi.org/10.1175/WAF1031.1.
Barnston, A. G., and S. J. Mason, 2011: Evaluation of IRI’s seasonal climate forecasts for the extreme 15% tails. Wea. Forecasting, 26, 545–554, https://doi.org/10.1175/WAF-D-10-05009.1.
Barnston, A. G., S. Li, S. J. Mason, D. G. DeWitt, L. Goddard, and X. Gong, 2010: Verification of the first 11 years of IRI’s seasonal climate forecasts. J. Appl. Meteor. Climatol., 49, 493–520, https://doi.org/10.1175/2009JAMC2325.1.
Batté, L., and M. Déqué, 2011: Seasonal predictions of precipitation over Africa using coupled ocean–atmosphere general circulation models: Skill of the ENSEMBLES project multimodel ensemble forecasts. Tellus, 63A, 283–299, https://doi.org/10.1111/j.1600-0870.2010.00493.x.
Becker, A., P. Finger, A. Meyer-Christoffer, B. Rudolf, K. Schamm, U. Schneider, and M. Ziese, 2013: A description of the global land-surface precipitation data products of the Global Precipitation Climatology Centre with sample applications including centennial (trend) analysis from 1901–present. Earth Syst. Sci. Data, 5, 71–99, https://doi.org/10.5194/essd-5-71-2013.
Bliefernicht, J., and A. Bárdossy, 2007: Probabilistic forecast of daily areal precipitation focusing on extreme events. Nat. Hazards Earth Syst. Sci., 7, 263–269, https://doi.org/10.5194/nhess-7-263-2007.
Bombardi, R. J., K. V. Pegion, J. L. Kinter, B. Cash, and J. M. Adams, 2017: Sub-seasonal predictability of the onset and demise of the rainy season over monsoonal regions. Front. Earth Sci., 5, https://doi.org/10.3389/feart.2017.00014.
Bouali, L., N. Philippon, B. Fontaine, and J. Lemond, 2008: Performance of DEMETER calibration for rainfall forecasting purposes: Application to the July–August Sahelian rainfall. J. Geophys. Res., 113, D15111, https://doi.org/10.1029/2007JD009403.
Braman, L. M., M. K. van Aalst, S. J. Mason, P. Suarez, Y. Ait-Chellouche, and A. Tall, 2013: Climate forecasts in disaster management: Red Cross flood operations in West Africa, 2008. Disasters, 37, 144–164, https://doi.org/10.1111/j.1467-7717.2012.01297.x.
Charles, A., B. Timbal, E. Fernandez, and H. Hendon, 2013: Analog downscaling of seasonal rainfall forecasts in the Murray Darling Basin. Mon. Wea. Rev., 141, 1099–1117, https://doi.org/10.1175/MWR-D-12-00098.1.
Chernick, M. R., 2008: Bootstrap Methods: A Guide for Practitioners and Researchers. 2nd ed. John Wiley and Sons, 369 pp.
Déqué, M., C. Dreveton, A. Braun, and D. Cariolle, 1994: The ARPEGE/IFS atmosphere model: A contribution to the French community climate modelling. Climate Dyn., 10, 249–266, https://doi.org/10.1007/BF00208992.
Diallo, I., M. B. Sylla, F. Giorgi, A. T. Gaye, and M. Camara, 2012: Multimodel GCM-RCM ensemble-based projections of temperature and precipitation over West Africa for the early 21st century. Int. J. Geophys., 2012, 972896, https://doi.org/10.1155/2012/972896.
Dieng, D., G. Smiatek, J. Bliefernicht, D. Heinzeller, A. Sarr, A. T. Gaye, and H. Kunstmann, 2017: Evaluation of the COSMO-CLM high-resolution climate simulations over West Africa. J. Geophys. Res. Atmos., 122, 1437–1455, https://doi.org/10.1002/2016JD025457.
Durre, I., M. J. Menne, B. E. Gleason, T. G. Houston, and R. S. Vose, 2010: Comprehensive automated quality assurance of daily surface observations. J. Appl. Meteor. Climatol., 49, 1615–1633, https://doi.org/10.1175/2010JAMC2375.1.
Dutra, E., F. Di Giuseppe, F. Wetterhall, and F. Pappenberger, 2013: Seasonal forecasts of droughts in African basins using the Standardized Precipitation Index. Hydrol. Earth Syst. Sci., 17, 2359, https://doi.org/10.5194/hess-17-2359-2013.
ECMWF, 2017: Seasonal Forecasting System. Accessed 7 December 2017, https://www.ecmwf.int/en/forecasts/documentation-and-support/long-range/seasonal-forecast-documentation/user-guide/seasonal-forecasting-system.
Efron, B., and G. Gong, 1983: A leisurely look at the bootstrap, the jackknife, and cross-validation. Amer. Stat., 37, 36–48, https://doi.org/10.1080/00031305.1983.10483087.
Efron, B., and R. J. Tibshirani, 1994: An Introduction to the Bootstrap. CRC Press, 459 pp.
Engel, T., A. H. Fink, P. Knippertz, G. Pante, and J. Bliefernicht, 2017: Extreme precipitation in the West African cities of Dakar and Ouagadougou: Atmospheric dynamics and implications for flood risk assessments. J. Hydrometeor., 18, 2937–2957, https://doi.org/10.1175/JHM-D-16-0218.1.
Fleury, L., and Coauthors, 2011: AMMA information system: An efficient cross-disciplinary tool and a legacy for forthcoming projects. Atmos. Sci. Lett., 12, 149–154, https://doi.org/10.1002/asl.303.
Folland, C., J. Owen, M. N. Ward, and A. Colman, 1991: Prediction of seasonal rainfall in the Sahel region using empirical and dynamical methods. J. Forecasting, 10, 21–56, https://doi.org/10.1002/for.3980100104.
Garric, G., H. Douville, and M. Déqué, 2002: Prospects for improved seasonal predictions of monsoon precipitation over Sahel. Int. J. Climatol., 22, 331–345, https://doi.org/10.1002/joc.736.
Gigerenzer, G., R. Hertwig, E. van den Broek, B. Fasolo, and K. V. Katsikopoulos, 2005: “A 30% chance of rain tomorrow”: How does the public understand probabilistic weather forecasts? Risk Anal., 25, 623–629, https://doi.org/10.1111/j.1539-6924.2005.00608.x.
Graham, R. J., and Coauthors, 2011: Long-range forecasting and the Global Framework for Climate Services. Climate Res., 47, 47–55, https://doi.org/10.3354/cr00963.
Hansen, J., S. Mason, L. Sun, and A. Tall, 2011: Review of seasonal climate forecasting for agriculture in sub-Saharan Africa. Exp. Agric., 47, 205–240, https://doi.org/10.1017/S0014479710000876.
Hogan, R. J., and I. B. Mason, 2012: Deterministic forecasts of binary events. Forecast Verification: A Practitioner’s Guide in Atmospheric Science, I. T. Jolliffe and D. B. Stephenson, Eds., John Wiley and Sons, 31–59.
Horton, P., M. Jaboyedoff, and C. Obled, 2017: Global optimization of an analog method by means of genetic algorithms. Mon. Wea. Rev., 145, 1275–1294, https://doi.org/10.1175/MWR-D-16-0093.1.
Jolliffe, I. T., and D. B. Stephenson, Eds., 2012: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. John Wiley and Sons, 274 pp.
Katz, R. W., and A. H. Murphy, 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, 222 pp.
Katz, R. W., A. H. Murphy, and R. Winkler, 1982: Assessing the value of frost forecasts to orchardists: A dynamic decision-making approach. J. Appl. Meteor., 21, 518–531, https://doi.org/10.1175/1520-0450(1982)021<0518:ATVOFF>2.0.CO;2.
Kirtman, B., and Coauthors, 2014: The North American Multimodel Ensemble: Phase-1 Seasonal-to-Interannual Prediction; Phase-2 toward Developing Intraseasonal Prediction. Bull. Amer. Meteor. Soc., 95, 585–601, https://doi.org/10.1175/BAMS-D-12-00050.1.
Klein, C., D. Heinzeller, J. Bliefernicht, and H. Kunstmann, 2015: Variability of West African monsoon patterns generated by a WRF multi-physics ensemble. Climate Dyn., 45, 2733–2755, https://doi.org/10.1007/s00382-015-2505-5.
Klein, C., J. Bliefernicht, D. Heinzeller, U. Gessner, I. Klein, and H. Kunstmann, 2017: Feedback of observed interannual vegetation change: A regional climate model analysis for the West African monsoon. Climate Dyn., 48, 2837–2858, https://doi.org/10.1007/s00382-016-3237-x.
Laprise, R., L. Hernández-Díaz, K. Tete, L. Sushama, L. Šeparović, A. Martynov, K. Winger, and M. Valin, 2013: Climate projections over CORDEX Africa domain using the fifth-generation Canadian Regional Climate Model (CRCM5). Climate Dyn., 41, 3219–3246, https://doi.org/10.1007/s00382-012-1651-2.
Lorenz, C., and H. Kunstmann, 2012: The hydrological cycle in three state-of-the-art reanalyses: Intercomparison and performance analysis. J. Hydrometeor., 13, 1397–1420, https://doi.org/10.1175/JHM-D-11-088.1.
Lorenz, E., 1969: Atmospheric predictability as revealed by naturally occurring analogues. J. Atmos. Sci., 26, 636–646, https://doi.org/10.1175/1520-0469(1969)26<636:APARBN>2.0.CO;2.
MacLachlan, C., and Coauthors, 2015: Global Sseasonal Forecast System version 5 (GloSea5): A high-resolution seasonal forecast system. Quart. J. Roy. Meteor. Soc., 141, 1072–1084, https://doi.org/10.1002/qj.2396.
Manzanas, R., 2017: Assessing the suitability of statistical downscaling approaches for seasonal forecasting in Senegal. Atmos. Sci. Lett., 18, 381–386, https://doi.org/10.1002/asl.767.
Maraun, D., and Coauthors, 2010: Precipitation downscaling under climate change: Recent developments to bridge the gap between dynamical models and the end user. Rev. Geophys., 48, RG3003, https://doi.org/10.1029/2009RG000314.
Mariotti, L., I. Diallo, E. Coppola, and F. Giorgi, 2014: Seasonal and intraseasonal changes of African monsoon climates in 21st century CORDEX projections. Climatic Change, 125, 53–65, https://doi.org/10.1007/s10584-014-1097-0.
Marty, R., I. Zin, C. Obled, G. Bontron, and A. Djerboua, 2012: Toward real-time daily PQPF by an analog sorting approach: Application to flash-flood catchments. J. Appl. Meteor. Climatol., 51, 505–520, https://doi.org/10.1175/JAMC-D-11-011.1.
Masih, I., S. Maskey, F. E. F. Mussá, and P. Trambauer, 2014: A review of droughts on the African continent: A geospatial and long-term perspective. Hydrol. Earth Syst. Sci., 18, 3635–3649, https://doi.org/10.5194/hess-18-3635-2014.
Mason, S. J., 2011: Seasonal forecasting using the Climate Predictability Tool (CPT). Science and Technology Infusion Climate Bulletin, Proc. 36th NOAA Annual Climate Diagnostics and Prediction Workshop, Fort Worth, TX, NOAA/National Weather Service, 180–182.
Mason, S. J., 2012: Seasonal and longer-range forecasts. Forecast Verification: A Practitioner’s Guide in Atmospheric Science, I. T. Jolliffe and D. B. Stephenson, Eds., John Wiley and Sons, 203–220.
Mason, S. J., and S. Chidzambwa, 2009: Position Paper: Verification of African RCOF Forecasts. Columbia University International Research Institute for Climate and Society Tech. Rep. 09-02, 26 pp., https://doi.org/10.7916/D85T3SB0.
Mason, S. J., and K. Tippett, 2017: Climate Predictability Tool version 15.5.10. International Research Institute for Climate and Society, Columbia University, https://doi.org/10.7916/D8G44WJ6.
Menne, M. J., I. Durre, R. S. Vose, B. E. Gleason, and T. G. Houston, 2012: An overview of the Global Historical Climatology Network-Daily database. J. Atmos. Oceanic Technol., 29, 897–910, https://doi.org/10.1175/JTECH-D-11-00103.1.
Molteni, F., and Coauthors, 2011: The new ECMWF seasonal forecast system (System 4). ECMWF Tech. Memo. 656, 49 pp., https://www.ecmwf.int/sites/default/files/elibrary/2011/11209-new-ecmwf-seasonal-forecast-system-system-4.pdf.
Moron, V., A. Robertson, and M. Ward, 2006: Seasonal predictability and spatial coherence of rainfall characteristics in the tropical setting of Senegal. Mon. Wea. Rev., 134, 3248–3262, https://doi.org/10.1175/MWR3252.1.
Mullan, A. B., and C. S. Thompson, 2006: Analogue forecasting of New Zealand climate anomalies. Int. J. Climatol., 26, 485–504, https://doi.org/10.1002/joc.1261.
Murphy, A. H., 1973: A new vector partition of the probability score. J. Appl. Meteor., 12, 595–600, https://doi.org/10.1175/1520-0450(1973)012<0595:ANVPOT>2.0.CO;2.
Murphy, A. H., 1997: Forecast verification. Economic Value of Weather and Climate Forecasts, R. W. Katz and A. H. Murphy, Eds., Cambridge University Press, 19–74.
Murphy, A. H., S. Lichtenstein, B. Fischhoff, and R. L. Winkler, 1980: Misinterpretations of precipitation probability forecasts. Bull. Amer. Meteor. Soc., 61, 695–701, https://doi.org/10.1175/1520-0477(1980)061<0695:MOPPF>2.0.CO;2.
Mwangi, E., F. Wetterhall, E. Dutra, F. Di Giuseppe, and F. Pappenberger, 2014: Forecasting droughts in Africa. Hydrol. Earth Syst. Sci., 18, 611–620, https://doi.org/10.5194/hess-18-611-2014.
National Oceanic and Atmospheric Administration, 2017: North American Multi-Model Ensemble. NCEP Climate Prediction Center, accessed 11 December 20017, www.cpc.ncep.noaa.gov/products/NMME/data.html.
Ndiaye, O., L. Goddard, and L. Ward, 2009: Using regional wind fields to improve general circulation model forecasts of July–September Sahel rainfall. Int. J. Climatol., 29, 1262–1275, https://doi.org/10.1002/joc.1767.
Ndiaye, O., M. Ward, and W. Thaiw, 2011: Predictability of seasonal Sahel rainfall using GCMs and lead-time improvements through the use of a coupled model. J. Climate, 24, 1931–1949, https://doi.org/10.1175/2010JCLI3557.1.
Nicholson, S. E., C. Funk, and A. H. Fink, 2018: Rainfall over the African continent from the 19th through the 21st century. Global Planet. Change, 165, 114–127, https://doi.org/10.1016/j.gloplacha.2017.12.014.
Nikulin, G., C. Jones, F. Giorgi, G. Asrar, M. Büchner,