1. Introduction
Operational dynamical seasonal forecasts of sea surface temperature (SST) have been produced by the Australian Bureau of Meteorology (hereinafter, the Bureau) for the Great Barrier Reef (GBR) since 2009 (Spillman and Alves 2009; Spillman et al. 2011; Spillman 2011b). Recently, monthly thermal stress forecasts have been developed for the western and central Pacific Ocean region (20°N–25°S, 120°E–150°W) as a part of the Pacific–Australia Climate Change Science and Adaptation Planning (PACCSAP) program (http://www.bom.gov.au/climate/pacific/projects.shtml), which is funded by the Australian Government’s Australian Aid program (now part of the Department of Foreign Affairs and Trade). This program worked with meteorological service agencies in 15 island nations in the western Pacific Ocean region (Fig. 1) over 18 months to develop local knowledge and tools for climate change impact assessment and disaster risk management (Cottrill et al. 2013).
The western-central Pacific region. Black dots show locations of coral reefs (ReefBase 2014). Red stars show three locations used for composite plots: A (0°, 190°E), B (0°, 150°E), and C (18°S, 148°E) (Figs. 6, 7, and 8, respectively).
Citation: Journal of Applied Meteorology and Climatology 55, 7; 10.1175/JAMC-D-15-0109.1
The island nations involved in the PACCSAP program were Timor-Leste, Palau, Federated States of Micronesia, Papua New Guinea (PNG), Marshall Islands, Nauru, Solomon Islands, Kiribati, Tuvalu, Vanuatu, Fiji, Samoa, Tonga, Niue, and Cook Islands (Fig. 1). These nations are very diverse in terms of their climates, geographies, natural resources, and cultures. Coral reefs are very important for many of these nations both culturally and economically, contributing through industries such as reef fisheries and tourism and providing coastal protection (Bryant et al. 1998; Sadovy 2005).
Coral reefs are highly productive and diverse ecosystems in otherwise relatively barren tropical oceans (Muscatine and Porter 1977; Hatcher 1988) and are particularly sensitive to the impacts of climate change (Glynn 1991; Riegl et al. 2011; Frieler et al. 2013). Coral bleaching is a stress response where symbiotic zooxanthellae are expelled from the coral tissues and can result in coral mortality and reduced coral cover (Glynn 1993; Hoegh-Guldberg 1999), which can take years to recover (Baker et al. 2008; Van Woesik et al. 2011). High water temperature is the primary cause of mass coral bleaching, which has been observed across all tropical oceans since the 1980s (Goreau 1990; Goreau and Hayes 1994; Wilkinson 1998; Berkelmans et al. 2004; Wilkinson and Souter 2008; Eakin et al. 2010). Other stress factors, such as pollution, high solar irradiance, and the presence of excessive silt can also cause bleaching, though usually on local scales (Glynn 1993; Brown 1997).
Coral bleaching severity is a function of the duration and magnitude of the thermal stress to which corals are exposed (Hoegh-Guldberg and Smith 1989; Glynn 1993). Goreau and Hayes (1994) suggested that exposure to elevated temperatures of 1°C above mean summer maxima for more than a month is required to induce bleaching. They defined a “hot spot” as the SST anomaly (SSTA) calculated relative to the warmest month of the monthly climatology [the maximum monthly mean (MMM)].
Mass coral bleaching events in tropical oceans around the world have been associated with El Niño–Southern Oscillation (ENSO) events because of widespread anomalously warm ocean temperatures that can be sustained for many months (Baker et al. 2008). In particular, the severe 1997/98 ENSO event caused widespread coral bleaching across the tropical oceans and mortality of 16% of the world’s reefs (Wilkinson 2004). ENSO is a basinwide coupled ocean–atmosphere interaction of winds, air pressure, and water temperature, and one of the dominant large-scale dynamical climate drivers in the Pacific region (Bjerknes 1969; Suarez and Schopf 1988). During an El Niño, weaker trade winds and reduced upwelling in the east result in warmer surface waters in the eastern or central Pacific. A La Niña results in the reverse pattern, with enhanced upwelling of cool water in the east and warmer waters accumulating in the west Pacific. An ENSO event tends to last for a year or two, with maximum SSTA usually occurring during the austral summer period (December–February).
The Predictive Ocean Atmosphere Model for Australia (POAMA) is the Bureau’s operational seasonal prediction system. It is a global dynamical coupled ocean–atmosphere model that simulates ocean and atmospheric climatic conditions up to nine months in advance. It is used to produce operational ENSO forecasts, GBR SST forecasts, and the Bureau’s official seasonal temperature and precipitation outlooks for Australia (http://www.bom.gov.au/climate/ahead/). Recently, monthly forecasts of potential bleaching conditions (SSTA and hot spots) have been developed for the western and central Pacific. Other studies have documented POAMA forecast skill for SST/ENSO (Wang et al. 2011; Spillman et al. 2011; Hudson et al. 2011; Griesser and Spillman 2012) and coral bleaching risk in the GBR and Caribbean (Spillman et al. 2011). Here, we examine SST and hot-spot forecast skill in the western and central Pacific in the context of coral bleaching risk and assess their potential value using a cost–loss ratio approach.
Advance warning of potential bleaching conditions can inform management decisions, potentially increasing reef resilience under climate change (Bellwood et al. 2004; Hughes et al. 2007; Maynard et al. 2009). POAMA coral bleaching risk forecasts for the GBR are an integral part of the early warning system in the GBR Marine Park Authority’s coral bleaching response (Spillman 2011b). Management strategies aim to reduce nonthermal stress factors, thereby minimizing the impacts of warm ocean temperatures on corals and providing improved conditions for postbleaching reef recovery. Possibilities include optimizing the deployment of limited resources (e.g., bleaching surveys, field studies, and baseline monitoring), limiting runoff from nearby land-based industries and developments, regulation of reef tourism and fishing industries, and community education and involvement campaigns (Maynard et al. 2009). As coral bleaching events are predicted to increase in frequency and severity in a warming climate, the development of early warning forecast systems is a part of a broad range of tools, techniques, and knowledge that will be required in order to safeguard the future of the world’s threatened coral reefs.
2. Method
a. Verification data
The verification dataset used in skill assessment is the United States’ National Oceanic and Atmospheric Administration (NOAA) Optimum Interpolation Sea Surface Temperature version-2 (OISST) data for 1982–2010 (Reynolds et al. 2002). Daily values at 0.25° × 0.25° spatial resolution were averaged over calendar months and interpolated onto the coarser POAMA ocean grid (2° × 0.5°–1.5°). Persistence forecasts, in which the most recent observations are persisted into the future, were created using observed monthly SSTA values. Persistence forecasts can provide a benchmark measure of minimum skill against which a forecast system can be judged (Spillman and Alves 2009).
b. Dynamical model description
POAMA is a global coupled ocean–atmosphere dynamical seasonal prediction system. The atmospheric component is the Bureau Atmospheric Model, version 3.0 (BAM; Colman et al. 2005), which has T47 horizontal resolution (approximately 250-km grid) and 17 vertical levels. The ocean component is the Australian Community Ocean Model (ACOM v2), which is based on the Modular Ocean Model produced by the Geophysical Fluid Dynamics Laboratory (Schiller et al. 1997). Ocean model grid spacing is 2° and 0.5°–1.5° in the zonal and meridional directions, respectively. There are 25 vertical levels, with 12 levels in the upper 200 m. The ocean–atmosphere coupler is the Ocean Atmosphere Sea Ice Soil system (OASIS v3; Valcke et al. 2000).
POAMA (version 2) consists of three submodels, each of which is run 11 times starting from slightly different initial conditions, for a total of 33 simulations per forecast (Wang et al. 2005; Wang et al. 2011; Hudson et al. 2013). Taken together, the 33 simulations (i.e., ensemble members) compose the forecast ensemble, and they provide an estimate of the uncertainty in the forecast. Probabilistic forecasts are derived using ensemble member distribution, while the ensemble mean forecast (which we use in this study) is the unweighted average of all 33 ensemble members. The initial conditions are produced by a coupled ensemble initialization scheme (Hudson et al. 2013): the ocean states are produced by the POAMA Ensemble Ocean Data Assimilation System (PEODAS; Yin et al. 2011), and the atmospheric state is produced by Atmosphere–Land Initialization (Hudson et al. 2011).
Retrospective forecasts (hindcasts) of monthly SST values for the verification period 1982–2010 (29 yr) have been used in this study. These hindcasts were initialized on the first of each month and were nine months in duration. Lead time is the time elapsed between the initialization date of the forecast and the month being forecast (the target month). For example, when a forecast is initialized on 1 February, the forecast lead for February is zero months; the forecast lead for March is one month, through to October, which is lead eight.
The monthly hindcast climatology is calculated over the period 1982–2010 and is a function of forecast initialization date and lead time. Each POAMA submodel has its own monthly climatology (Hudson et al. 2013). Monthly model SSTA forecasts were generated by removing the hindcast climatology from the monthly SST forecasts.
c. Coral bleaching risk metrics
The MMM SST is the SST of the warmest month of the observed monthly climatology for 1982–2010 (Goreau and Hayes 1994; Liu et al. 2003) at each grid cell. Monthly hot spots are calculated by subtracting the MMM from the SST, where only positive anomalies are of significance (in practice, negative anomalies are set to zero). These were calculated for bias-corrected model forecasts (model SSTA + observed monthly climatology), for all ensemble members and the ensemble mean. Hotspots were also calculated for observed SST values and persistence SST forecasts. For hot-spot calculations, model SST values were bias corrected by adding the observed (OISST) monthly climatology to model SSTA forecasts for the same period. This provides a basic correction for model mean state bias (Spillman 2011a).
d. Skill measures
Forecast skill was assessed using continuous and categorical skill metrics and calculated at each grid cell for all months and lead times and for both model and persistence forecasts. We present all categorical skill scores for a lead time of one month.
First, the Pearson correlation coefficient r indicates the strength of the phase relationship between observed and forecast values, where a value of 1 indicates a perfect association [although with the possibility of errors of location (i.e., bias) and amplitude]. The correlation coefficient does not reflect biases and, further, can be significantly affected by the presence of outliers; hence, if it is to be used, it must be accompanied by approaches that do characterize errors of location and amplitude. The correlation coefficient was calculated between ensemble mean forecast SST and observed SST values. Correlations from all cells in the western-central Pacific Ocean (20°N–25°S, 120°E–150°W; Fig. 1) were then averaged.
Second, forecast skill was assessed using root-mean-square (RMS) error, which provides quantitative information about the magnitude of the forecast error (i.e., the difference between the forecast value and the observed value). The RMS error was normalized by the standard deviation of the observed values, resulting in a nondimensional skill measure. A normalized RMS error (NRMSE) of 1 indicates that the mean forecast error has a magnitude equal to the variability in the observed signal; such a forecast often has little or no value. NRMSE values significantly smaller than 1 are therefore desirable.
Model variability was assessed by comparing the standard deviations of forecast and observed SSTA values. For the model, the standard deviation of SSTA for each individual ensemble member was calculated. These values were then averaged in order to assess the amplitude of the simulated intrinsic variability, which may be suppressed in the ensemble mean signal.
To assess hot-spot forecast skill, the correlation coefficient and RMS error are unsuitable because of the rarity of hot-spot occurrence. Instead, we use various categorical forecast skill measures, which score the forecast system based on its ability to predict the occurrence of a specific event. An event is defined here as the occurrence of a hot-spot value above zero, whereas a nonevent means a hot-spot value of zero. Hits, misses, false alarms, and correct negatives are defined as shown in Table 1.
Contingency table. The significance of the boldface text is described in the text.
Categorical skill scores calculated include hit rate, proportion correct, false alarm rate, false alarm ratio, frequency bias, and Peirce skill score (Table 2; Wilks 2011; Mason 2012). The Peirce skill score (PSS) includes all four values in the contingency table (Table 1; boldface text), whereas the other scores do not and so inevitably have limitations or biases, as noted in the rightmost column of Table 2. Taken together, however, this set of six scores provides comprehensive information about the most important aspects of forecast quality. The climatological frequency is the proportion of months in the verification period during which a hot spot > 0.2°C is observed; the model has its own climatological frequency, which is the fraction of months in which the event is predicted by the model. These categorical skill scores do not take into account hot-spot magnitude, which is an important factor in coral bleaching.
e. Teleconnections
At three locations (A, B, and C; Fig. 1), both the observed and the model hot-spot time series data were extracted.
For each hot-spot time series, months during which the hot-spot value exceeded a threshold of 0.2°C were then used to create SSTA composite maps to examine instantaneous teleconnections across the Pacific. This 0.2°C threshold excluded the weakest hot spots (thereby providing a clearer composited result), while retaining enough independent events to provide statistical significance to the results. An independent event comprises only those consecutive months during which the hot-spot threshold is exceeded at a particular location. Although all months that exceeded the threshold were included in the composites, the level of statistical significance was calculated based only on the number of independent events. The three locations were selected to include regions of good and poor skill (Fig. 5) and with differing ENSO influences (Trenberth and Stepaniak 2001).
Also shown are months during which the (observed) Niño-3.4 index exceeded 0.8°C (green highlighting; Figs. 7, 8, and 9). The Niño-3.4 index is defined as the SSTA averaged over the region 5°N–5°S and 170°–120°W (Trenberth and Stepaniak 2001). Months during which the Niño-3.4 index exceeded 0.8°C are shown in order to investigate relationships between hot spots and the ENSO cycle, which is considered to be the primary source of interannual variability of SST in the Pacific (Bjerknes 1969; Suarez and Schopf 1988).
f. Value analysis
The cost–loss ratio decision model, as described by Richardson (2012), is a method of assessing forecast value that considers the cost–loss ratio, the hit rate, the false alarm rate, and the observed climatological frequency (how often the event happens in reality). In a simplified situation, the occurrence of a binary (yes/no) event incurs a loss L, unless action is taken that costs C and entirely prevents the loss (Table 3). We assume that any action taken completely negates any loss L, though in reality some loss may still be incurred even when action has been taken.
Expense matrix showing cost C incurred when action is taken and loss L incurred in the absence of preventative measures (Richardson 2012).
3. Results
a. SST skill
The average correlation skill of the model exceeds that of persistence forecasts at all lead times; r = 0.81 and 0.73 for the model and persistence forecasts, respectively, at lead 0. Model skill decreases to 0.43 at lead 8, and persistence forecast skill is no longer statistically significant from lead 4 (two-tailed t test, r < 0.37, p = 0.05, and n = 29; Fig. 2a).
(a) Average correlation coefficient and (b) average NRMSE of SST forecasts over the western-central Pacific region (Fig. 1) for the model (black circles) and persistence (gray diamonds) for all months in 1982–2010. In (a), the horizontal dashed line corresponds to a statistically significant correlation at p = 0.05 with n = 29 (two-tailed t test). In (b), the horizontal solid line indicates the level of no skill.
Citation: Journal of Applied Meteorology and Climatology 55, 7; 10.1175/JAMC-D-15-0109.1
The normalized RMS error of the model remains below 1 for lead 0–8 months, while, for persistence forecasts, the normalized RMS error exceeds 1 from lead 2 months onward (Fig. 2b), indicating no skill.
Standard deviations of the monthly forecast SSTA at lead 1 for all months in 1982–2010 are presented in Fig. 3a, together with the difference with observed SSTA standard deviations (Fig. 3b; forecast standard deviation minus observed standard deviation). The high interannual variability along the equator is clearly visible (Fig. 3a) and is too wide in the model (red shading in Fig. 3b). The model appears to underrepresent the large variability seen across the upwelling region toward the eastern side of the Pacific basin (blue shading in Fig. 3b).
(a) Standard deviation of monthly forecast SSTA at lead time 1 month over 1982–2010, and (b) difference between (a) and standard deviation of observed monthly SSTA for 1982–2010.
Citation: Journal of Applied Meteorology and Climatology 55, 7; 10.1175/JAMC-D-15-0109.1
b. Hot-spot forecast skill
Categorical skill scores for model and persistence forecasts for monthly hot-spot occurrences (greater than 0°C) averaged across the western-central Pacific are shown in Fig. 4 (see Table 2). The proportion of correct model forecasts is only marginally higher (~0.04) than that of persistence forecasts in predicting the occurrence of hot spots (Fig. 4a). The hit rate of the model is approximately the same as that of the persistence forecast (Fig. 4b). However, the model’s false alarm rate is under 10% at all lead times, indicating that, of all observed nonevents, less than 10% were incorrectly predicted by the model to be events (i.e., false alarms; Fig. 4c). The false alarm ratio is approximately 30% at lead zero, indicating that of all predicted events, 30% were incorrectly predicted as events when in reality they were nonevents (i.e., false alarms; Fig. 4d). The persistence forecast achieves close to the ideal frequency bias ratio of 1, while the model shows a marked drop with lead time, indicating that the model is not forecasting events frequently enough on average (Fig. 4e). The Peirce skill score (Fig. 4f), which includes consideration of all four values in the contingency table (Table 1), is higher for the model than for the persistence forecasts at all lead times, approximately 0.6 at lead 0 and decreasing to 0.37 at lead 8.
Categorical skill scores for hot-spot occurrences, averaged over the western-central Pacific region: (a) proportion correct, (b) hit rate, (c) false alarm rate, (d) false alarm ratio, (e) frequency bias, and (f) Peirce skill score (see Table 2 for definitions).
Citation: Journal of Applied Meteorology and Climatology 55, 7; 10.1175/JAMC-D-15-0109.1
While the area averaged scores provide a broad indication as to how forecast skill in the central-western Pacific changes with forecast lead time, skill also varies spatially (Fig. 5). The spatial patterns of most skill measures, shown for lead 1, are roughly similar in the study region: generally good skill along the equator in the central Pacific and relatively low skill northeast of PNG and also at latitudes greater than approximately 10°S and 10°N. At longer lead times, the spatial distributions of skill for all metrics appear similar to those at lead 1, though skill is degraded (not shown).
Categorical forecast skill scores for prediction of hot-spot events at lead 1: (a) proportion correct, (b) hit rate, (c) false alarm rate, (d) false alarm ratio, (e) frequency bias, and (f) Peirce skill score (see Table 2 for definitions). In (a)–(d) and (f), pale shading denotes good forecast performance; dark colors denote poor forecast performance.
Citation: Journal of Applied Meteorology and Climatology 55, 7; 10.1175/JAMC-D-15-0109.1
c. Teleconnections
The time series of observed and predicted (lead 1) hot-spot events at location A (central equatorial Pacific) are shown in Fig. 6a. Clearly evident is a series of strong hot-spot events, some lasting for up to 12 months (e.g., 1997/98). Almost all hot-spot events at this location coincide with the timing of an El Niño event, defined as Niño-3.4 SSTA > 0.8°C (Fig. 6a; green shading). This is to be expected, considering that location A is on the boundary of the Niño-3.4 region (Trenberth and Stepaniak 2001). At this location, observed monthly hot-spot values exceeded 0.2°C (horizontal dark line in Fig. 6a) for 117 months, comprising 24 independent events. The SSTA composite of these months (Fig. 6b) shows a significant (p < 0.02; n = 24) positive SST signal along the equator eastward all the way to South America, corresponding broadly to the typical El Niño SST pattern. The equivalent composite for the model SSTA using the same months (at lead 1; Fig. 6c) is very similar (assisted by the good match between the model’s hot-spot time series and the observed), though the equatorial warm region is broader in the central and eastern Pacific, with peak values shifted westward, when compared with the observed SSTA composite.
(a) Time history of observed and model (lead 1) hot spots at location A (0°, 190°E; green star), where green shading indicates when the Niño-3.4 SST index exceeded 0.8°C. (b) Mean observed SSTA composite map for months exceeding observed hot-spot threshold of 0.2°C; (c) as in (b), but for model SSTAs at lead 1. Stippled areas are significantly different from zero at p < 0.02, with n = 24 (two-tailed t test) for both (b) and (c).
Citation: Journal of Applied Meteorology and Climatology 55, 7; 10.1175/JAMC-D-15-0109.1
In comparison with location A, hot-spot events at location B (western equatorial Pacific) were more frequent (66 months where hot spot > 0.2°C; 34 independent events) but of shorter duration (Fig. 7a). There also appears to be little correspondence with El Niño events (Fig. 7a; green bars) in either the observed or predicted hot-spot time series (red and gray lines). The model appears to have captured many of the observed hot-spot events (proportion correct is above 0.7), though several model events begin too early (e.g., at the start of 1989 and in late 1985, 2000, and 2009). The model missed the particularly high values of the observed peaks between 2002 and 2004, and it entirely missed the events of 1987, 1998, and 2003 (all three of which occurred or began during ENSO events and were relatively short).
(a)–(c) As in Fig. 6, but for location B (0°, 150°E; green star). (d) Mean observed SSTA composited over months during which the model hot spots at lead 1 exceeded 0.2°C. In (b)–(d), stippled areas are significantly different from zero at p < 0.02 with n = 34, 34, and 26 independent events, respectively (two-tailed t test).
Citation: Journal of Applied Meteorology and Climatology 55, 7; 10.1175/JAMC-D-15-0109.1
The corresponding observed SSTA spatial composite is statistically significantly different from zero along the equator in the central and eastern Pacific (Fig. 7b; stippled; P < 0.02, with n = 34) and over a region extending southeast from the far-west equatorial Pacific. However, when compared with the observed SSTA composite, the model SSTA composite (Fig. 7c) shows significant negative anomalies along the equator, extending farther westward to the date line, and a reduced signal in the positive anomalies in the west and southwest Pacific. When the set of months that define the composite is based on those months during which model hot-spot values exceeded 0.2°C rather than using observed hot-spot values, however, a clearer pattern emerges in the model SSTA composite (Fig. 7d). West and southwest Pacific model SSTA are of similar magnitude to the observed values shown in Fig. 7b.
The time series of hot-spot values at location C (in the GBR) and corresponding SSTA composites are shown in Fig. 8. The observed hot-spot time series (red line) exceeds the 0.2°C threshold in 30 months, making up 16 independent events, as compared with 24 and 34 independent events at locations A and B, respectively. Model hot spots (gray line) exceed the threshold for just 18 months, making up 13 independent events. The events tend to be of short duration (usually less than 3 months) and limited to the austral summer months (December–February) at this off-equatorial location, primarily because of the pronounced seasonal SST cycle, which makes hot spots outside the warm summer months unlikely. At this location, no SSTA pattern exists along the equator in the observed composite (Fig. 8b), indicating no simultaneous relationship between hot-spot events and ENSO. The model SSTA composite (Fig. 8c) does not show the localized (Coral Sea) warming seen in the observations (Fig. 8b). The model SSTA composite based on the model hot-spot time series (13 events) is shown in Fig. 8d.
As in Fig. 7, but for location C (18°S, 148°E; green star). In (b)–(d), stippled areas are significantly different from zero at p < 0.02 with n = 16, 16, and 13, respectively (two-tailed t test).
Citation: Journal of Applied Meteorology and Climatology 55, 7; 10.1175/JAMC-D-15-0109.1
d. Forecast value
For the three locations A, B, and C, Table 4 lists the hot-spot forecast hit rates and false alarm rates at lead 1 and the climatological frequencies for the period 1982–2010 (348 months in total).
Hit rates, false alarm rates, and observed climatological frequencies at three locations.
Figure 9 shows the forecast value as a function of cost–loss ratio calculated for each location using the parameters listed in Table 4. Negative values are not shown, as these forecasts are of no value. The horizontal axis spans cost–loss ratios between 0 and 1; if the cost–loss ratio exceeds one, the cost of taking action would exceed the cost of the loss that is to be prevented. A perfect forecast would achieve a value of 1 on the vertical axis for all cost–loss ratios, meaning that the average expense would be minimized (i.e., equal to Eperfect). The peak of each curve occurs at a cost–loss ratio that is equal to the observed climatological hot-spot frequency at that location. The forecast value at each peak is equal to the hit rate minus the false alarm rate, which is the definition of the Peirce skill score.
Forecast value for the three locations: A (solid line), B (short dashes), and C (long dashes) as a function of cost–loss ratio.
Citation: Journal of Applied Meteorology and Climatology 55, 7; 10.1175/JAMC-D-15-0109.1
At location A, there is value in the lead 1 forecasts for cost–loss ratios between approximately 0.1 and 0.8, whereas for locations B and C the range of positive forecast values is considerably smaller, reflecting the lower forecast skill in those areas (Fig. 5). The maximum forecast value at location A is 0.65, whereas at location B it is 0.4. These values are consistent with the shading shown in Fig. 5f.
All of the curves go to zero toward the lower (left) end of the horizontal (cost–loss ratio) axis, because at low cost–loss ratios it becomes viable to ignore the forecasts and simply take action all the time; because the hit rate is less than 1, using the forecasts would result in the occurrence of missed events incurring large losses. The curve for location C (Fig. 9, long dashes) extends farthest to the left since, at location C, the false alarm rate is lower than at locations A and B (Fig. 5c).
The forecast system also loses value as the cost–loss ratio approaches 1. This occurs because the loss incurred by missing an event is not much higher than the cost of taking action, and when considered together with the “wasted” expense associated with false alarms, it becomes viable to not use the forecasts, never take action, and simply accept the losses.
4. Discussion and conclusions
One of the primary purposes of an operational forecast system is to provide value to users by facilitating more informed decision-making. The economic value of a forecast system can be assessed when the various costs of action and inaction are known, together with the hit rate, false alarm rate, and climatological frequency (Richardson 2012). This study has calculated these skill scores over the western and central Pacific region, allowing users to quantify the potential value of incorporating POAMA forecasts for coral bleaching risk in their decision-making processes.
The correlation coefficient and normalized RMS error scores for POAMA SSTA forecasts show that the model outperforms persistence forecasts at all lead times (Fig. 2), when averaged over the study area (Fig. 1). Averaging the skill score over the area provides information about the skill at the grid scale, which can be quite different to the skill associated with predicting an areally averaged quantity, such as an El Niño SST index. Furthermore, while we use such an average to make broad comparisons of model performance with persistence, it is not necessarily appropriate as a way to estimate forecast value for a particular local application, because there is considerable spatial variability in model skill.
The exaggerated variability of model SSTA in the off-equatorial central Pacific (around latitudes of ±2°; Fig. 3) is indicative of a wider-than-observed region of cool upwelling, a phenomenon also seen in several other climate models (Van Oldenborgh et al. 2005; AchutaRao and Sperber 2006; Vannière et al. 2013). This effect is also evident in Fig. 6c, where the model SSTA composite is broader than the corresponding observed composite, even though hot-spot timing is well forecast. In the equatorial eastern Pacific, the model’s deficiency in SSTA variability (Fig. 3) interestingly appears to have almost no detrimental effect on the categorical skill scores, at least when calculated with a threshold of 0°C [and also 0.5°C (not shown)]. This is a welcome result.
POAMA hot-spot forecasts generally outperform persistence forecasts at all lead times when averaged over the study area (Fig. 4), except in frequency bias. Since frequency bias is a statistical property of a time series that takes no account of event timing, and the persistence forecast at a particular lead is simply a time-shifted copy of the observed time series, the frequency bias of a persistence forecast is expected to achieve the ideal ratio of 1. This score illustrates why it is important to be aware of the limitations and potential biases of skill scores. This reduction in the model’s frequency bias as a function of lead time is likely (at least partly) the result of the use of the ensemble mean value in conjunction with a binary criterion (above/below a threshold): as the ensemble spread increases with lead time, more members are likely to fall below the hot-spot temperature threshold, reducing (on average) the frequency of hot-spot events.
Considering now the spatial distribution of forecast skill, by most measures (Table 2) the hot-spot forecasts have good skill within approximately 10° of the equator in the central Pacific and less toward the far east and far west (Fig. 5).These skill distributions reflect the influence of ENSO dynamics on SSTA distributions to some extent, with POAMA owing considerable predictability to this slow quasi-oscillatory phenomenon (Barnett et al. 1993; Shukla et al. 2000; Zhao et al. 2011). The influence of ENSO is also evident in SST composites shown in Figs. 7–9.
Hit rates are highest (approximately 0.8) in the central equatorial Pacific Ocean, while low false alarm rates (less than 0.2) occur over most of the tropical Pacific Ocean (96% of grid cells between latitudes 15°S and 15°N); higher false alarm rates are confined to a region near PNG. Hit rate and false alarm rate are used in the value assessment: a high hit rate and a low false alarm rate will result in high forecast value.
At location A (in the central Pacific), hot spots are long-lived (up to 10 months) and intense (up to 2°C in observations). Hot spots almost always coincide with ENSO events (Fig. 6a; gray line and green shading), and the SSTA composite associated with hot-spot events shows a typical ENSO pattern (Trenberth and Stepaniak 2001). Skill here is high by most measures (Fig. 5), and the forecasts have value over a broad range of cost–loss ratios (Fig. 9, solid line).
At location B (in the western Pacific), hot spots are weaker than at location A, though more frequent. There appears to be little correspondence between hot spots and ENSO events (Fig. 7a; gray line and green shading). The model variability in this region is approximately correct (Fig. 3b), and the model produces about the right number of hot-spot months (frequency bias ≈ 1; Fig. 5e). Further, when produced using the model’s hot-spot time series, the model composite (Fig. 7d) shows similar patterns to the observed composite (Fig. 7b), indicating that physical teleconnections over the Pacific Ocean to this location are simulated and that the behavior of the ocean when a hot-spot event is occurring is well reproduced. It is therefore largely the model’s inability to accurately predict the timing of hot spots that causes the relatively low skill scores in this location. Forecasting the timing at this location may be challenging because hot spots here are often of short duration and are not limited to any particular time of year (as there is no significant seasonal cycle). As a consequence of the low skill scores, the potential forecast value is lower, and the cost–loss ratio above which forecasts have no value is considerably reduced (0.4, as compared with 0.8 at location A).
At location C, on the Great Barrier Reef, hot-spot events show little relationship with ENSO (Fig. 8a); bleaching episodes have been observed in both active ENSO and ENSO-neutral years (Berkelmans et al. 2004; Diaz-Pulido et al. 2009). At this location, the relatively low skill scores appear to be due to a tendency to predict too few hot-spot months rather than an inability to accurately forecast their timing (which is made easier by the strong seasonality). At lead 1, the frequency bias is 0.7 (ideally 1), meaning that too few events were forecast, and the hit rate is 0.46 (ideally 1), meaning that more than half of observed events were not predicted. More positive is that the false alarm rate is very low: below 0.05 (ideally 0), meaning that very few forecast events did not happen (Fig. 5c). In addition to predicting too few hot spots, the model also appears to underpredict the magnitude of many of the observed hot-spot peaks (Fig. 8a) (e.g., 1999, 2002, 2004, 2005, and 2006). It is therefore interesting that the model’s SSTA variability nonetheless matches observations quite well (Fig. 3). Potential forecast value at location C is greater than at location B over a broader range of cost–loss ratios despite the relatively low hit rate, largely because the false alarm rate is very low (0.05).
Furthermore, the observed SSTA composite (Fig. 8b) is similar to the model SSTA composite, which uses model hot spots > 0.2°C (Fig. 8d), but not the model SSTA composite, which uses observed hot spots > 0.2°C (Fig. 8c). As was the case for the location B analysis, this indicates that when the model predicts hot spots above 0.2°C, it has done so by accurately simulating ocean conditions that tend to coincide with observed conditions during observed hot spots (i.e., for the correct physical reasons). This does not, however, provide any guidance as to why the model fails to reproduce these teleconnections at the correct times.
A simple comparison between the standard deviations of the forecast and observed monthly SST values (Fig. 3) indicates where there may be potential for improving hot-spot skill with techniques such as inflation-of-variance calibration (Johnson and Bowler 2009). Such calibration can help to correct amplitude errors by ensuring that the range of forecast values (the variance) matches the range of values in the observational record. In the region around PNG and the Australian tropical coastline, where skill is low, the differences in standard deviation are relatively small, suggesting that there is little potential for inflation-of-variance calibration technique to improve the categorical skill scores of hot-spot forecasts. The small difference in standard deviation is consistent with our earlier finding that low skill around the equatorial western Pacific (location B) was due largely to errors in timing, not in amplitude. More fundamental improvements to the model itself, and to the observing systems that provide its initial conditions, may be more likely to yield improvements in skill and, hence, forecast value.
Particularly encouraging is that, despite the obvious region of lower skill in the far-western equatorial Pacific (northeast of PNG), which is evident in Figs. 5a and 5c, forecasts in this area can still have value, depending on how much management actions cost relative to the expense incurred when a hot-spot event is missed. In fact, the Peirce skill score is above 0.2 in almost the whole study region (Fig. 5f; more than 99% of grid cells between latitudes 15°S and 15°N) and since it is equal to the peak value of the forecast value curve, there is potentially some value in the hot-spot forecasts almost everywhere.
Coral reef ecosystems around the world are now under unprecedented pressure from a variety of stressors, including overfishing and destructive fishing practices, pollution from marine and land sources, coastal development, invasive species, disease, and temperature-related bleaching (Hughes and Connell 1999; Pandolfi et al. 2011). In the face of these threats, rigorous understanding of coral ecosystems and effective management tools and practices will be critical (though perhaps not sufficient) to ensure the long-term survival of these important natural treasures. At a local scale, various activities are under investigation or in use for their potential ability to assist in reducing bleaching intensity, including shading of reefs, enhancing the feeding ability of coral polyps, electrical stimulation, artificial upwelling to cool surface waters, and the implementation of temporary fishing no-take zones (Baker et al. 2008). When these activities can be executed within the timeframes of weeks, or even a few months, there is scope for the kind of forecasts examined herein to be relevant in the planning process. One would ideally apply the methods we have presented herein to specific local conditions and management options (including, e.g., consideration of lead time). Knowing where and when forecast systems such as POAMA may be able to contribute to the effectiveness of management activities can be important for the uptake and ongoing use of such systems. In relation to dealing with the threat of widespread bleaching, however, there are limited intervention actions available.
The skill scores presented herein, in addition to the forecasts themselves, are available on a project funded research web portal (http://poama.bom.gov.au/experimental/pasap/sst.shtml). It displays an interactive map which can be viewed at various scales and panned to the region of interest and that shows the latest POAMA forecasts of SST, SSTA, and hot spots. At the time of writing, the portal is an experimental research product without 24/7 operational support.
Acknowledgments
This research contributes to the Pacific Australia Climate Change Science and Adaptation Program (PACCSAP) funded by AusAID and the Department of Climate Change and Energy Efficiency and delivered by the Bureau of Meteorology and CSIRO. We thank Robert Fawcett and Brad Murphy for reviewing this paper and Griffith Young, Guomin Wang, and Faina Tseitkin for preparation of the POAMA hindcasts. Coral reef location data were provided by ReefBase (http://www.reefbase.org).
REFERENCES
AchutaRao, K., and K. R. Sperber, 2006: ENSO simulation in coupled ocean–atmosphere models: Are the current models better? Climate Dyn., 27, 1–15, doi:10.1007/s00382-006-0119-7.
Baker, A. C., P. W. Glynn, and B. Riegl, 2008: Climate change and coral reef bleaching: An ecological assessment of long-term impacts, recovery trends and future outlook. Estuarine Coastal Shelf Sci., 80, 435–471, doi:10.1016/j.ecss.2008.09.003.
Barnett, T. P., N. Graham, S. Pazan, W. White, M. Latif, and M. Flügel, 1993: ENSO and ENSO-related predictability. Part I: Prediction of equatorial Pacific sea surface temperature with a hybrid coupled ocean–atmosphere model. J. Climate, 6, 1545–1566, doi:10.1175/1520-0442(1993)006<1545:EAERPP>2.0.CO;2.
Bellwood, D. R., T. P. Hughes, C. Folke, and M. Nyström, 2004: Confronting the coral reef crisis. Nature, 429, 827–833, doi:10.1038/nature02691.
Berkelmans, R., G. De’ath, S. Kininmonth, and W. J. Skirving, 2004: A comparison of the 1998 and 2002 coral bleaching events on the Great Barrier Reef: Spatial correlation, patterns, and predictions. Coral Reefs, 23, 74–83, doi:10.1007/s00338-003-0353-y.
Bjerknes, J., 1969: Atmospheric teleconnections from the equatorial Pacific. Mon. Wea. Rev., 97, 163–172, doi:10.1175/1520-0493(1969)097<0163:ATFTEP>2.3.CO;2.
Brown, B. E., 1997: Coral bleaching: Causes and consequences. Coral Reefs, 16, S129–S138, doi:10.1007/s003380050249.
Bryant, D., L. Burke, J. McManus, and M. Spalding, 1998: Reefs at Risk: A Map-Based Indicator of Threats to the World’s Coral Reefs. World Resources Institute, 56 pp.
Colman, R., and Coauthors, 2005: BMRC atmospheric model (BAM) version 3.0: Comparison with mean climatology. BMRC Research Rep. 108, 66 pp.
Cottrill, A., and Coauthors, 2013: Seasonal forecasting in the Pacific using the coupled model POAMA-2. Wea. Forecasting, 28, 668–680, doi:10.1175/WAF-D-12-00072.1.
Diaz-Pulido, G., and Couathors, 2009: Doom and boom on a resilient reef: Climate change, algal overgrowth and coral recovery. PLoS One, 4, e5239, doi:10.1371/journal.pone.0005239.
Eakin, C. M., and Coauthors, 2010: Caribbean corals in crisis: Record thermal stress, bleaching, and mortality in 2005. PLoS One, 5, e13969, doi:10.1371/journal.pone.0013969.
Frieler, K., M. Meinshausen, A. Golly, M. Mengel, K. Lebek, S. D. Donner, and O. Hoegh-Guldberg, 2013: Limiting global warming to 2°C is unlikely to save most coral reefs. Nat. Climate Change, 2, 165–170, doi:10.1038/nclimate1674.
Glynn, P. W., 1991: Coral reef bleaching in the 1980s and possible connections with global warming. Trends Ecol. Evol., 6, 175–179, doi:10.1016/0169-5347(91)90208-F.
Glynn, P. W., 1993: Coral reef bleaching: Ecological perspectives. Coral Reefs, 12, 1–17, doi:10.1007/BF00303779.
Goreau, T. J., 1990: Coral bleaching in Jamaica. Nature, 343, 417–417, doi:10.1038/343417a0.
Goreau, T. J., and R. L. Hayes, 1994: Coral bleaching and ocean “hot spots.” Ambio, 23, 176–180.
Griesser, A. G., and C. M. Spillman, 2012: SST forecast skill of the new intra-seasonal configuration of POAMA-2. CAWCR Res. Lett., 8, 10–16. [Available online at http://www.cawcr.gov.au/researchletters/CAWCR_Research_Letters_8.pdf.]
Hatcher, B. G., 1988: Coral reef primary productivity: A beggar’s banquet. Trends Ecol. Evol., 3, 106–111, doi:10.1016/0169-5347(88)90117-6.
Hoegh-Guldberg, O., 1999: Climate change, coral bleaching and the future of the world’s coral reefs. Mar. Freshwater Res., 50, 839–866, doi:10.1071/MF99078.
Hoegh-Guldberg, O., and G. J. Smith, 1989: The effect of sudden changes in temperature, light and salinity on the population density and export of zooxanthellae from the reef corals Stylophora pistillata Esper and Seriatopora hystrix Dana. J. Exp. Mar. Biol. Ecol., 129, 279–303, doi:10.1016/0022-0981(89)90109-3.
Hudson, D. A., O. Alves, H. H. Hendon, and G. Wang, 2011: The impact of atmospheric initialisation on seasonal prediction of tropical Pacific SST. Climate Dyn., 36, 1155–1171, doi:10.1007/s00382-010-0763-9.
Hudson, D. A., A. G. Marshall, Y. Yin, O. Alves, and H. H. Hendon, 2013: Improving intraseasonal prediction with a new ensemble generation strategy. Mon. Wea. Rev., 141, 4429–4449, doi:10.1175/MWR-D-13-00059.1.
Hughes, T. P., and J. H. Connell, 1999: Multiple stressors on coral reefs: A long-term perspective. Limnol. Oceanogr., 44, 932–940, doi:10.4319/lo.1999.44.3_part_2.0932.
Hughes, T. P., and Coauthors, 2007: Phase shifts, herbivory, and the resilience of coral reefs to climate change. Curr. Biol., 17, 360–365, doi:10.1016/j.cub.2006.12.049.
Johnson, C., and N. Bowler, 2009: On the reliability and calibration of ensemble forecasts. Mon. Wea. Rev., 137, 1717–1720, doi:10.1175/2009MWR2715.1.
Liu, G., A. E. Strong, and W. Skirving, 2003: Remote sensing of sea surface temperatures during 2002 Barrier Reef coral bleaching. Eos, Trans. Amer. Geophys. Union, 84, 137–141, doi:10.1029/2003EO150001.
Mason, I. B., 2012: Binary events. Forecast Verification: A Practitioner’s Guide in Atmospheric Science, I. T. Jolliffe and D. B. Stephenson, Eds., John Wiley and Sons, 37–73.
Maynard, J. A., J. E. Johnson, P. A. Marshall, C. M. Eakin, G. Goby, H. Schuttenberg, and C. M. Spillman, 2009: A strategic framework for responding to coral bleaching events in a changing climate. Environ. Manage., 44, 1–11, doi:10.1007/s00267-009-9295-7.
Muscatine, L., and J. W. Porter, 1977: Reef corals: Mutualistic symbioses adapted to nutrient-poor environments. BioScience, 27, 454–460, doi:10.2307/1297526.
Pandolfi, J. M., S. R. Connolly, D. J. Marshall, and A. L. Cohen, 2011: Projecting coral reef futures under global warming and ocean acidification. Science, 333, 418–422, doi:10.1126/science.1204794.
ReefBase, 2014: ReefBase: A global information system for coral reefs. [Available online at http://www.reefbase.org].
Reynolds, R. W., N. A. Rayner, T. M. Smith, D. C. Stokes, and W. Wang, 2002: An improved in situ and satellite SST analysis for climate. J. Climate, 15, 1609–1625, doi:10.1175/1520-0442(2002)015<1609:AIISAS>2.0.CO;2.
Richardson, D. S., 2012: Economic value and skill. Forecast Verification: A Practitioner’s Guide in Atmospheric Science, 2nd ed. I. T. Jolliffe and D. B. Stephenson, Eds., John Wiley and Sons, 165–186.
Riegl, B. M., S. J. Purkis, A. S. Al-Cibahy, M. A. Abdel-Moati, and O. Hoegh-Guldberg, 2011: Present limits to heat-adaptability in corals and population-level responses to climate extremes. PLoS One, 6, e24802, doi:10.1371/journal.pone.0024802.
Sadovy, Y., 2005: Trouble on the reef: The imperative for managing vulnerable and valuable fisheries. Fish Fish., 6, 167–185, doi:10.1111/j.1467-2979.2005.00186.x.
Schiller, A., J. S. Godfrey, P. C. McIntosh, and G. A. Meyers, 1997: A global ocean general circulation model for climate variability studies. CSIRO Marine Laboratories Rep. 227, 60 pp.
Shukla, J., and Coauthors, 2000: Dynamical seasonal prediction. Bull. Amer. Meteor. Soc., 81, 2593–2606, doi:10.1175/1520-0477(2000)081<2593:DSP>2.3.CO;2.
Spillman, C. M., 2011a: Operational real-time seasonal forecasts for coral reef management. J. Oper. Oceanogr., 4, 13–22, doi:10.1080/1755876X.2011.11020119.
Spillman, C. M., 2011b: Advances in forecasting coral bleaching conditions for reef management. Bull. Amer. Meteor. Soc., 92, 1586–1591, doi:10.1175/BAMS-D-11-00065.1.
Spillman, C. M., and O. Alves, 2009: Dynamical seasonal prediction of summer sea surface temperatures in the Great Barrier Reef. Coral Reefs, 28, 197–206, doi:10.1007/s00338-008-0438-8.
Spillman, C. M., O. Alves, and D. A. Hudson, 2011: Seasonal prediction of thermal stress accumulation for coral bleaching in the tropical oceans. Mon. Wea. Rev., 139, 317–331, doi:10.1175/2010MWR3526.1.
Suarez, M. J., and P. S. Schopf, 1988: A delayed action oscillator for ENSO. J. Atmos. Sci., 45, 3283–3287, doi:10.1175/1520-0469(1988)045<3283:ADAOFE>2.0.CO;2.
Trenberth, K. E., and D. P. Stepaniak, 2001: Indices of El Niño evolution. J. Climate, 14, 1697–1701, doi:10.1175/1520-0442(2001)014<1697:LIOENO>2.0.CO;2.
Valcke, S., L. Terray, and A. Piacentini, 2000: OASIS 2.4 ocean atmospheric sea ice soil users guide, version 2.4. CERFACS Tech. Rep. TR/CMGC/00/10.
Van Oldenborgh, G. J., S. Y. Philip, and M. Collins, 2005: El Niño in a changing climate: A multi-model study. Ocean Sci., 1, 81–95, doi:10.5194/os-1-81-2005.
Van Woesik, R., K. Sakai, A. Ganase, and Y. Loya, 2011: Revisiting the winners and the losers a decade after coral bleaching. Mar. Ecol. Prog. Ser., 434, 67–76, doi:10.3354/meps09203.
Vannière, B., E. Guilyardi, G. Madec, F. J. Doblas-Reyes, and S. Woolnough, 2013: Using seasonal hindcasts to understand the origin of the equatorial cold tongue bias in CGCMs and its impact on ENSO. Climate Dyn., 40, 963–981, doi:10.1007/s00382-012-1429-6.
Wang, G., O. Alves, and N. Smith, 2005: BAM3.0 tropical surface flux simulation and its impact on SST drift in a coupled model. BMRC Research Rep. 107, 30 pp.
Wang, G., D. A. Hudson, Y. Yin, O. Alves, H. H. Hendon, S. Langford, G. Liu, and F. Tseitkin, 2011: POAMA-2 SST skill assessment and beyond. CAWCR Res. Lett., 6, 40–46. [Available online at http://www.cawcr.gov.au/researchletters/CAWCR_Research_Letters_6.pdf.]
Wilkinson, C. R., 1998: Status of coral reefs of the world: 1998. Australian Institute of Marine Science Rep., 181 pp. [Available online at http://epubs.aims.gov.au/handle/11068/3320.]
Wilkinson, C. R., 2004: Status of coral reefs of the world: 2004. Australian Institute of Marine Science Rep., 572 pp. [Available online at http://epubs.aims.gov.au/handle/11068/6840.]
Wilkinson, C. R., and D. Souter, Eds., 2008: Status of Caribbean coral reefs after bleaching and hurricanes in 2005. Global Coral Reef Monitoring Network and Reef and Rainforest Research Centre Rep., 152 pp. [Available online at http://www.coris.noaa.gov/activities/caribbean_rpt/.]
Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. International Geophysics Series, Vol. 100, Academic Press, 704 pp.
Woodcock, F., 1976: The evaluation of yes/no forecasts for scientific and administrative purposes. Mon. Wea. Rev., 104, 1209–1214, doi:10.1175/1520-0493(1976)104<1209:TEOYFF>2.0.CO;2.
Yin, Y., O. Alves, and P. R. Oke, 2011: An ensemble ocean data assimilation system for seasonal prediction. Mon. Wea. Rev., 139, 786–808, doi:10.1175/2010MWR3419.1.
Zhao, M., G. Wang, H. H. Hendon, and O. Alves, 2011: Impact of including surface currents on simulation of Indian Ocean variability with the POAMA coupled model. Climate Dyn., 36, 1291–1302, doi:10.1007/s00382-010-0823-1.
