1. Introduction
Medium-range predictability and forecastability1 of low-frequency modes of atmospheric variability can be evaluated more readily with a long time series of forecasts. In this manuscript we evaluate two such modes of variability—Northern Hemispheric atmospheric blocking and the Madden–Julian oscillation (MJO; Madden and Julian 1971)—and their interactions using an extensive set of global medium-range ensemble reforecasts.
Both the MJO and blocking occur somewhat infrequently at a particular location. Using the blocking definition of Tibaldi and Molteni (1990) averaged over several decades the Northern Hemisphere wintertime, blocking frequency ranges from approximately 2% to 22%, depending on the longitude. Blocks are most common over the eastern Atlantic Ocean and western Europe, with a secondary frequency maximum in the central Pacific Ocean. Blocks can be persistent, leading to both long periods of rather similar, and often high-impact weather. For the slowly moving, large-scale, equatorially trapped tropical convective envelopes and associated wind perturbations known as the MJO, a given year may produce only a couple coherent, high-amplitude events, although this number will vary depending on the definition of the MJO (Straub 2013).
Evaluating the predictability and forecastability of blocking and the MJO are challenging given the infrequency and the temporal continuity of these events, the latter of which reduces the effective sample size [Wilks 2006, Eq. (5.12)]. If one is interested, for example, in the ability of the model to forecast block onset for a particular longitude band, a 3-month period may have, say, 15 days with blocked conditions, but most likely those 15 days occurred in 1–3 persistent blocking events. Similar issues occur with the MJO. The interaction of two such events is even more difficult to evaluate with limited samples (e.g., evaluating the change in blocking frequency related to a large forecast MJO event in the Indian Ocean). A large sample size could be provided from a reforecast (i.e., a multiyear or preferably a multidecadal sample of forecasts from a fixed forecast model and assimilation system).
Extensive reforecasts (hindcasts) often have not been available to facilitate such studies. For blocking, the most comprehensive recent study was by Jung et al. (2012), which used hindcasts to document blocking frequency in extended-range simulations from the European Centre for Medium-Range Weather Forecasts (ECMWF) model at various resolutions. The authors found that Euro-Atlantic sector blocking frequency was generally more underforecast with lower-resolution models. Older studies included the Watson and Colucci (2002) study of Northern Hemispheric wintertime blocking using data from the operational National Centers for Environmental Prediction (NCEP) global spectral model from 1995 to 1998, and Mauritsen and Källén (2004), who studied blocking in the ECMWF system during the Northern Hemispheric 2000/01 winter. Both studies found too few blocks in the forecast. Pelly and Hoskins (2003a), using a potential vorticity–based method of defining blocks, evaluated the ECMWF output for a year of data beginning on 1 August 2001, a period spanning several model changes. Though they also found blocking frequency was underforecast, there was positive skill in the probabilistic forecasts of blocks out to 10 days, and they found that block onset was a better forecast than block cessation.
There have been more recent studies of the MJO than for blocking, for seasonal simulations at least. The MJO is being actively studied in part because it affects monsoon (Yasunari 1979) and tropical cyclone variability (Maloney and Hartmann 2000a,b). The MJO can also excite extratropical Rossby wave trains (Knutson and Weickmann 1987; Jones et al. 2004; Weickmann and Berry 2009) and can interact with midlatitude, low-frequency modes of variability such as the North Atlantic Oscillation (L'Heureux and Higgins 2008; Cassou 2008; Lin et al. 2009). The MJO is often poorly forecast, and there is some evidence that an improved MJO forecast may result in improved midlatitude forecasts (Ferranti et al. 1990; Vitart and Molteni 2010).
The interaction of the MJO with the midlatitude flow and its forecastability has been an active area of investigation (Liebmann and Hartmann 1984; Kiladis and Weickmann 1992; Hendon et al. 2000; Riddle et al. 2013). Since the MJO represents the variability at time scales of 30–70 days, (e.g., Waliser et al. 2009), it has been common to examine numerical simulations to lead times of several months and to leverage hindcasts to provide large-enough samples. Examples of studies with seasonal forecast models and hindcasts include Hendon et al. (2000), Lin et al. (2008), Seo et al. (2009), Kim et al. (2009), Vitart and Molteni (2010), Gottschalck et al. (2010), Kang and Kim (2010), Jia et al. (2010), and Crueger et al. (2013). For the medium range (here, roughly from +3 to +16 days of lead time), the literature on MJO forecast evaluation is sparser. Some verification statistics were calculated from The Observing System Research and Predictability Experiment (THORPEX) Interactive Grand Global Ensemble (TIGGE; Bougeault et al. 2010) data by Matsueda and Endo (2011).
Recently, National Oceanic and Atmospheric Administration (NOAA) scientists created an extensive global ensemble reforecast dataset (Hamill et al. 2013) using the version of the NCEP Global Ensemble Forecast System (GEFS) that was operational in 2012–13. This dataset was created to facilitate the diagnosis and statistical correction of systematic forecast errors in medium-range ensemble forecasts, thereby improving GEFS guidance. In this article, we demonstrate an important ancillary purpose, showing how the extensive reforecasts facilitate the diagnosis of errors in low-frequency modes of variability, modes that can have a profound impact on intraseasonal forecast skill. Specifically, we will examine the usefulness of the reforecasts for examining the predictability and forecastability of blocking and the MJO as well as their interrelationships. The overarching hypothesis, which as we will show is easily disproved, is that the ensemble prediction system well represents the evolution of forecast uncertainty of these phenomena (i.e., the ensemble forecasts and observed can be considered random draws from the same underlying distribution).
Below, section 2 briefly describes the dataset and the methods for forecast evaluation. Section 3 provides results, while section 4 provides a discussion and conclusions.
2. Data and methods
a. Description of the datasets
Unless noted otherwise, reforecast data for the December–February 1985–2012 period were used in this study. The global ensemble reforecast data were more completely described in Hamill et al. (2013). Briefly, this reforecast dataset is based on the 2012 version of the NCEP GEFS. An 11-member retrospective ensemble forecast was generated to +16-day lead for every day from 0000 UTC 1 December 1984 to the current date. Consistent with the operational GEFS, the model resolution was T254L42 to day +8 (~40-km grid spacing at 40° latitude and 42 levels). Starting at day +7.5 and extending to day +16, the reforecasts were conducted at the reduced resolution of T190L42 (~54-km grid spacing). Through 20 February 2011, the Climate Forecast System Reanalysis (CFSR; Saha et al. 2010) provided the control initialization and verification. Thereafter, the operational gridpoint statistical interpolation (GSI; Kleist et al. 2009) procedure was used, which was updated to a hybrid variational-ensemble data assimilation approach (Hamill et al. 2011) on 22 May 2012. Additional ensemble member perturbed initial conditions were generated using the ensemble transform with rescaling approach of Wei et al. (2008). See Hamill et al. (2013) for more details on the dataset, including a description of the extensive amount of data that are available for fast-access download. In this study, data interpolated to a 1° grid were used.
For examination of associated tropical precipitation forecasts, 1° Global Precipitation Climatology Project (GPCP; Huffman et al. 2001) data were used. This GPCP data were available only from 1997 to the current period.
b. Blocking and MJO definitions
For blocking, though there have been some modern alternatives (e.g., Pelly and Hoskins 2003b; Barnes et al. 2012), here blocking was defined directly following the simple method of Tibaldi and Molteni (1990) based on 500-hPa geopotential heights. Conditional climatologies of blocked and unblocked 500-hPa height patterns at the international date line are shown in Fig. 1. Given the increased frequency of blocking in the Euro-Atlantic and Pacific regions (Fig. 2a), our analysis of blocking forecastability and predictability was limited to two sectors, the “Euro-Atlantic” sector, from 45°W longitude to 45°E longitude, and the “Pacific” sector, from 140°E longitude to 130°W longitude.
Composite of Northern Hemisphere 500-hPa geopotential height patterns under (a) blocked flow at 180° longitude and (b) unblocked flow at 180° longitude.
Citation: Monthly Weather Review 142, 2; 10.1175/MWR-D-13-00199.1
(a) Blocking frequency as determined from analyses and reforecasts as a function of forecast lead time. Areas shaded in gray denote the two sectors in subsequent figures, the Pacific and Euro-Atlantic sectors. (b) Analyzed, spatially smoothed yearly blocking frequencies for each year between 1985 and 2012.
Citation: Monthly Weather Review 142, 2; 10.1175/MWR-D-13-00199.1
For MJO analysis, a now-standard (Gottschalck et al. 2010) method was used to compute the projections onto the two leading empirical orthogonal functions (EOFs) of MJO variability (Wheeler and Hendon 2004, hereafter WH04). The respective EOFs are commonly known as the first and second all-season Real-time Multivariate MJO Indices (RMM1 and RMM2). (The EOF structures associated with RMM1 and RMM2 were taken directly from the real-time MJO website, http://cawcr.gov.au/staff/mwheeler/maproom/RMM/.) The EOFs were computed from a time series of longitudinal arrays of filtered anomalies in the 200- and 850-hPa zonal winds and outgoing longwave radiation (OLR), which were averaged in the band from 15°S to 15°N latitude and normalized by their variances over this latitude band and all longitudes. This filtering also mostly removed interannual variability and the projection onto the El Niño–Southern Oscillation (ENSO). The processing of the reforecast data to calculate the projections of forecast data onto RMM1 and RMM2 generally followed the procedure outlined by WH04, with the following exception. To attempt to remove the effects of ENSO, the projection of the filtered data onto the Niño-3.4 index anomalies was removed (Trenberth 1997), as opposed to the “SST1” index of Drosdowsky and Chambers (2001) cited in WH04. The filtering did remove the leading three harmonics of the annual cycle and the mean of the previous 120 days, as in WH04. Our procedure, when applied to the control initial condition from the CFSR, produced a time series of projections that correlated at ~0.87 to the time series produced by WH04 with NCEP–National Center for Atmospheric Research (NCAR) reanalyses (Kalnay et al. 1996) data (not shown). The discrepancies are most likely due to the reduced quality of the NCEP–NCAR reanalysis relative to the CFSR analysis, and the use of observed OLR in the RMMs calculated from NCEP–NCAR data and reanalysis OLR from the CFSR.
c. Methods of forecast evaluation
Many methods were used to evaluate blocking and MJO forecast skill. For blocking, a Brier skill score (BSS) was used that measured the skill of the ensemble forecast's ability to forecast the probability of blocked conditions. The Brier score of the forecast and climatology were computed separately for each longitude in the standard manner [Eq. (7.34) in Wilks (2006)]. The BSS [Eq. (7.35) in Wilks (2006)] was computed from the sums of Brier scores of the forecast and climatology over each longitude within a sector. Forecast probabilities of blocking were set directly by ensemble relative frequency. For example, if 3 of the 11 members were diagnosed as having blocked conditions at a particular longitude, the forecast probability was set to 
Reliability diagrams (Wilks 2006, p. 287) for blocking probabilities were also calculated in the standard manner.
We were also interested in the ability of the ensemble to provide high-quality, reliable probabilistic guidance. For the MJO, reliability (or more accurately, “consistency”) was evaluated with rank histograms (Hamill 2001). As the rank histograms for RMM1 and RMM2 were similar, the average of the two was reported. Rank histograms were also generated for MJO phase and amplitude. In generating the rank histograms, the observed value of RMM1 and RMM2 were assumed to be perfect, so no noise was introduced to ensemble members to potentially account for inaccuracies in the analyses of RMMs (Fig. 6 in Hamill 2001). The phase and amplitude propagation characteristics of the reforecasts will also be examined; the specific methodology for these will be discussed at the relevant point in the results section.
3. Results
a. Blocking forecasts
Figure 2a shows the blocking frequency in the GEFS reforecasts for selected forecast lead times. Overall, the GEFS replicated blocking frequency reasonably accurately, though for lead times of +6 days and beyond it underforecasted blocking frequency in the Euro-Atlantic sector by up to 25%. While the blocking frequency curves are relatively smooth over the multidecadal period, this disguises tremendous interannual variability. Figure 2b shows the yearly blocking frequencies, spatially smoothed slightly to aid in interpretability. For a given longitude, blocking frequencies can vary by an order of magnitude or more from one year to the next. Figure 3 shows that overall positive blocking forecast skill was retained through day +13, but this skill was far short of the skill that was possible under perfect-model assumptions. For example, the perfect-model skill at day +7 was as large as the actual skill at ~day +3.5. It is somewhat likely that the perfect-model estimate of forecast skill is somewhat too large, too, due to the tendency for the ensemble forecasts to cluster together, such that their spread is not statistically consistent with their ensemble-mean error (e.g., Bougeault et al. 2010; Figs. 2–3). Figure 3 above also shows that blocking onset and cessation were somewhat less well forecast than the overall forecasts of blocking. The skill curves for onset and cessation were noisier because of the greatly reduced sample size, even with 28 winter seasons of data. Figure 4 shows that there was a substantial amount of variability in the skill of blocking when the data were sorted by half-decadal periods, plus 2010–12. In the Pacific sector, the blocking skill in the most recent 3 years was the largest, but the 2005–09 period was intermediate in skill and actually comparable to the skill during the 1985–89 period. However, in the Euro-Atlantic sector, the skill for the 1985–89 period was substantially smaller than for the subsequent half decades. The dashed lines in Fig. 4 provide the half-decadal skill results under the perfect-model assumption. These show some natural variability in skill at half-decadal time scales. Note that the 1985–89 period, for example, had the lowest perfect-model predictability in the Euro-Atlantic sector, which probably contributed to its especially low real-model skill, while the 2010–12 period had among the highest perfect-model skill in the Pacific, indicating that the actual high forecast skill for this period was in part natural variability. The perfect-model results here also suggest that even in the best of circumstances, blocking forecast skill as defined here is limited to approximately two weeks.
Brier skill scores of blocking probability forecasts for the (a) Pacific and (b) Euro-Atlantic sectors.
Citation: Monthly Weather Review 142, 2; 10.1175/MWR-D-13-00199.1
As in Fig. 3, but for Brier skill scores of blocking probability forecasts by half decade for the (a) Pacific and (b) Euro-Atlantic sectors. Solid lines present the skill scores for the actual reforecasts, dashed lines present the skill under perfect-model assumptions.
Citation: Monthly Weather Review 142, 2; 10.1175/MWR-D-13-00199.1
Figure 5 presents reliability diagrams for the blocking forecasts at various lead times. At the earlier lead times the blocking forecasts are mostly reliable, but the reliability decreases so that by day +15 the forecasts are rather unreliable. As expected, blocking forecast sharpness decreases over time, as seen in the usage frequency histograms. There are probably many reasons for the lack of reliability, including all the usual suspects with unreliable ensemble forecasts; the moderate resolution of the forecast model, the deficiencies in parameterizations, including sometimes inappropriate deterministic formulations (Palmer 2012), the suboptimal initialization of both the control and perturbations in the ensemble system, and, as we shall see below, faulty representations of interactions with the MJO.
Reliability diagrams for blocking probability forecasts for (a) +3-day forecast, (b) +6-day forecast, (c) +9-day forecast, (d) +12-day forecast, and (e) +15-day forecast. Dotted red line denotes the skill in the Euro-Atlantic sector and dotted blue line denotes the skill in the Pacific sector. Red and blue bars indicate the frequency of usage of each forecast probability category for the Euro-Atlantic and Pacific sectors, respectively.
Citation: Monthly Weather Review 142, 2; 10.1175/MWR-D-13-00199.1
b. MJO forecasts
Figure 6 shows the evolution from both analyzed and deterministic forecasts where the vector (RMM1, RMM2) of initial conditions was within 0.5 units of (1.5, −1.5), that is, within the purple circle on the figure. The samples are all various dates from December–February 1985–2012, and samples that were less than 5 days apart from another sample were eliminated. It appears that the collection of forecasts propagate somewhat more regularly than the collection of analyzed states, and perhaps the forecasts lose some amplitude. Figures 7 and 8 attempt to quantify this, providing a PDF of the daily change in overall RMM angle and magnitude, respectively, as well as the change attributable specifically to wind and OLR components of the RMM. The cases used to populate Figs. 7 and 8 were selected in an attempt to isolate situations where there was a real MJO between Africa and the Maritime Continent, and where it had robust associated convection. First, we note that a given RMM vector can be decomposed into a component due to the OLR and a component due to the winds. Consequently, a subset of December–February 1985–2012 cases was selected that met the following criteria: (i) an overall RMM amplitude of greater than 1.0, (ii) an RMM2 < 0.0, and (iii) an amplitude of the OLR component >0.5. There were 90 such dates. Consider the overall phase change in Fig. 7a; after a reasonable mean phase change the first day, for subsequent leads the mean phase change was substantially smaller for the forecast than for the analyzed, with a phase change of 5°–10° day−1 for the analyzed but 3°–6° for the forecast. Forecast MJOs propagated too slowly, on average, a problem noted also in the CFS version 2 (CFS v2; Fu et al. 2013). The forecast propagation of the OLR component of RMM was slower than the wind component, though this was true to a lesser extent with analyzed data as well. This slower propagation can also be diagnosed, for example, from the reanalyzed rainfall and wind lag correlations shown in Fig. 3 from Weaver et al. (2011). The overall RMM phase change distribution also more closely resembles the wind change than the OLR change, evidence of the domination of RMM calculation by the wind component (Straub 2013). By inspection, the distributions of phase changes of the forecast PDFs were not dramatically narrower than of the analyzed PDFs; the main deficiency was a biased mean, not a lack of spread. Considering the overall magnitude change in Fig. 8a, both forecast and analyzed exhibit a similar small decrease in magnitude, though the PDF for the forecast appears slightly more narrow and peaked than for the analyzed.
RMM1 and RMM2 phase plots for (a) analyzed and (b) control forecasts whose initial states are within the purple circle. Differences in time from the initial time conveyed by the colors of lines and dots, with the legend indicating the lead time in days. Phases 1–8 are marked in the corners of the diagram.
Citation: Monthly Weather Review 142, 2; 10.1175/MWR-D-13-00199.1
PDF of daily change in angle of RMM vector (Δθ), measured in degrees, for (a) overall RMM, (b) OLR contribution to RMM, and (c) wind-component contribution to RMM. Only December–February 1985–2012 dates that had an initial RMM magnitude of greater than 1.0, an RMM2 component <0.0, and an initial OLR component of >0.5 were included as samples. Dots indicate the mean of the PDF for a given day, horizontally offset slightly so that dots do not overlap.
Citation: Monthly Weather Review 142, 2; 10.1175/MWR-D-13-00199.1
As in Fig. 7, but for the PDF of the change in magnitude of RMM.
Citation: Monthly Weather Review 142, 2; 10.1175/MWR-D-13-00199.1
Figure 9 provides more evidence of the deficiencies of phase and amplitude in the MJO forecasts. Here, for Figs. 9a,b, the set of dates in December–February 1985–2012 was identified that had an (RMM1, RMM2) in phase 1 (i.e., −180° ≤ θ ≤ −135°). Figures 9c,d use initial dates with an (RMM1, RMM2) in phase 4 (i.e., −45° ≤ θ ≤ −0°). The panels then present the lagged OLR and 850-hPa wind component anomalies during the subsequent 15 days, using analyzed data (Figs. 9a,c) and forecast data (Figs. 9b,d). For a comparison to CFS v2, see Fig. 9 of Wang et al. (2013). Consider first the analyzed data for phase 1 in Fig. 9a. Here, a small initial MJO convective anomaly was roughly centered over Africa. Fifteen days later a stronger cold anomaly had developed that was centered more over the western Indian Ocean. Low-level zonal wind convergence anomalies (as diagnosed from the 0.0 wind contour) moved from ~30°E at the initial time to ~100°E 15 days later. Considering the associated forecast data in Fig. 9b, the initial cold OLR anomaly and the wind anomaly did not grow during the subsequent 15 days, nor did the anomalies propagate as rapidly; the forecast model generally misses the initiation of the MJO, and in this regard performs worse than CFS v2 (Wang et al. 2013). Figure 9c presents the analyzed anomaly data for phase 4, when the center of the convection associated with MJO is approaching the Maritime Continent. The maintenance and propagation of cold OLR anomalies can be clearly seen, as well as the gradual shift of low-level convergence toward the international date line during the subsequent 15 days. Considering the associated forecast data (Fig. 9d), here the MJO-associated convection and the strength of forecast wind anomalies were better maintained relative to phase 1 forecasts. However, the forecast wind anomalies still propagated too slowly, as diagnosed from the movement of the 0 anomaly contour to only ~150°E during the subsequent 15 days.
Composite of lagged filtered OLR anomalies (shaded) and 850-hPa u-component anomalies (contours, m s−1) subsequent to initial large MJOs (RMM amplitude >1.0). (a) Analyzed, initial MJO in phase 1; (b) forecast, initial MJO in phase 1; (c) analyzed, initial MJO in phase 4; and (d) forecast, initial MJO in phase 4. 133 cases were used in (a) and (c) and 214 cases were used in (b) and (d).
Citation: Monthly Weather Review 142, 2; 10.1175/MWR-D-13-00199.1
Not only did the forecast MJO wind and precipitation features propagate too slowly, but also the precipitation forecasts exhibited significant unconditional bias (Fig. 10). Precipitation amounts were dramatically overforecast at the early forecast leads. The general pattern of the daily GPCP precipitation amount climatology was reasonably replicated in day 0 to +1 forecast (Fig. 10b), but the average daily forecast precipitation amounts were commonly >50% too large. By the beginning of the second week of the forecast (Fig. 10c), the overforecast bias was reduced, but there was less resemblance with the analyzed precipitation pattern. For example, the connection of the South Pacific convergence zone (SPCZ; Folland et al. 2002) to the intertropical convergence zone was missing, and the forecast SPCZ was unduly zonally oriented, as it often is in climate simulations (Brown et al. 2011). These pattern changes and an excess of forecast precipitation in the central to eastern Pacific were apparent at the end of week +2 (Fig. 10d).
December–February 1997–2012 daily-average precipitation climatologies for (a) analyzed GPCP data, (b) +0–1-day forecast, (c) +7–8-day forecast, and (d) +15–16-day forecast.
Citation: Monthly Weather Review 142, 2; 10.1175/MWR-D-13-00199.1
An additional deficiency of the probabilistic MJO forecasts was their underdispersion and/or conditional bias. This can be seen by examining the rank histograms from the ensemble predictions of MJO RMMs (Fig. 11). All of the rank histograms were U shaped, which was most pronounced for the short-lead RMM amplitude forecasts. The rank histograms indicate that there was unrealistic consistency of magnitudes among the forecasts; the ensemble prediction system did not adequately simulate the forecast processes that contribute to diversity in MJO magnitudes.
Rank histograms of RMM1 and RMM2 values (green bars), the angle of the vector in the (RMM1, RMM2) phase space (red bars), and the magnitude (blue bars), for forecast lead times of 1–16 days.
Citation: Monthly Weather Review 142, 2; 10.1175/MWR-D-13-00199.1
Despite the significant biases, the forecasts still exhibit skill in the first week. Figure 12 presents the CRPSS of the forecasts measured relative to an unconditional climatology and relative to the regression-based lagged persistence model. The lagged persistence model presented a tougher reference standard, so forecasts exhibited less skill in comparison to this. Skill diminished to near 0 by day +11 with respect to lagged persistence and by day +14 with respect to climatology. While lagged persistence represented a tougher reference, we note that the first-generation Climate Forecast System at NCEP (Wang et al. 2005; Saha et al. 2006) produced forecasts that had higher errors and less correlation skill than the lagged persistence at all forecast leads. Thus, the current GEFS provides substantial improvement in the simulation of the MJO relative to the first-generation CFS (CFS v1).
CRPSS of ensemble reforecasts of the MJO relative to an unconditional climatological distribution (red line) and relative to a lagged regression model using current and recent analyzed RMM values as predictors (blue line). The green line shows the “perfect model” skill, when one forecast member is used as a surrogate for the verification and the remaining 10 members are used to generate the probabilities.
Citation: Monthly Weather Review 142, 2; 10.1175/MWR-D-13-00199.1
Consider now the correlation skill and RMSE of MJO forecasts (Fig. 13). Overall correlation skill and RMSE were comparable to those from the more accurate models shown in Matsueda and Endo (2011), though not as large as for the CFS v2 (Wang et al. 2013, see their Fig. 3). Perhaps this difference is in part due to the use of a coupled ocean–atmosphere in the CFS v2, unlike in the GEFS used here. There was more correlation skill in the wind components of the RMM than in the OLR component, as shown in Fig. 13a. Note that the Wheeler–Hendon RMM index is dominated by its wind component (Straub 2013). The greater skill for wind likely relates to the better ability of models to maintain and evolve the rotational component of the wind over the globe, perhaps due to improved initial conditions. The correlation skill was much lower for the first two half-decadal periods, 1985–89 and 1990–94, than for the subsequent periods, with the exception of 2010–12, though the RMS error was not larger for the first decade of the forecast (Figs. 13c,d). The generally greater skill since 2000 is likely due in part to assimilating a greater number and variety of satellite observations, especially radiance data from polar-orbiting satellites (Gelaro et al. 2010). Figures 13e,f show that forecasts initialized from analyses with high amplitudes of RMM exhibited more skill but also higher RMSE than lower-amplitude forecasts. This was also shown in Lin et al. (2008, see their Fig. 13) for the Canadian models.
(a),(c),(e) Correlation skill and (b),(d),(f) RMSE for MJO forecasts. (a),(b) Skill and RMSE for total and for individual wind and OLR components of RMM. (c),(d) Overall skill and RMSE for half-decadal periods. (e),(f) Skill for cases with initial large and small amplitude, as defined in the text.
Citation: Monthly Weather Review 142, 2; 10.1175/MWR-D-13-00199.1
c. Interactions between blocking and the MJO
Finally, we briefly consider the ability of the forecast model to successfully replicate the ability to discern changes in blocking frequency for different phases of strong MJOs. Strong MJOs are defined as the set of dates where the magnitude of the RMM is in the upper quartile of its distribution, forecast or observed. Figure 14 presents the results. Consider Fig. 14a. Here the change in blocking frequency for a strong MJO relative to the unconditional blocking frequency is shown as a function of longitude (abscissa) and of the analyzed phase θ (ordinate) of the MJO, as defined in section 2c. To provide an adequate sample size for a given θ, the data plotted for a given θ actually includes analyzed samples with similarly diagnosed θ, specifically where −22.5° ≤ θ ≤ 22.5°. Note some interesting characteristics in the analyzed relationship of blocking frequency changes. As θ varies between 0° and 120° (i.e., the MJO's center moves from the Maritime Continent to the Western Hemisphere), at 0° longitude, the blocking frequency changes from a strongly negative anomaly in blocking frequency to a strongly positive anomaly. Restated, analyzed Euro-Atlantic blocking frequency changes from below its long-term average to above average as the MJO moves east from the Maritime Continent. We note that this is consistent with previous results, such as the 500-hPa anomaly composite for various phases of the MJO in Lin et al. (2009, see their Fig. 4). Our Figs. 14b–d then show the respective blocking frequency anomalies when the forecast MJO phase is of the noted angle, and when the forecast MJO magnitude is greater than the upper quartile of the forecast distribution. The day +4 forecast in Fig. 14b still replicates many of the essential anomalies of the analyzed, including the shift from a strongly positive blocking frequency anomaly to a strongly negative anomaly along the Greenwich Meridian as θ varies between 0° and 120°. In some regards, this may not fully represent a particular ability of the forecast model to predict the interaction, but may also be an artifact of the persistent nature of blocks. Much of the frequency anomaly detail is lost by day +8, and the day +16 forecasts show no apparent relation to the analyzed. From this, we can conclude that the internal dynamics of the GEFS do not represent very well the processes that lead to interrelationships between blocking and the MJO.
Change in blocking frequency as a function of longitude for (a) when analyzed or (b) +4-, (c) +8-, or (d) +16-day forecast has RMM of phase theta ±22.5°. Phases of MJO diagram as marked in Fig. 6 are noted on the right-hand side of the plots. The ranges of the Euro-Atlantic and Pacific sectors are noted with heavy black lines at the bottom of each panel.
Citation: Monthly Weather Review 142, 2; 10.1175/MWR-D-13-00199.1
4. Discussion and conclusions
Some modes of atmospheric variability are uncommon enough and/or operate on long-enough time scales that a short time series of past forecasts will not prove sufficient for diagnosing their characteristics. Atmospheric blocking and the MJO are two such phenomena. In this paper we have shown how a very long time series of ensemble forecast guidance facilitates a greater understanding of the forecastability and predictability of these phenomena. In this case, the long time series was provided by a 28-yr dataset of reforecasts from the NCEP Global Ensemble Forecast System. The paper more specifically explored Northern Hemispheric blocking, the MJO, and their interaction during the December–February 1985–2012 period.
With regards to blocking, the reforecasts showed that the GEFS slightly underforecasted blocking frequency at longer leads in the Euro-Atlantic sector. Furthermore, the interannual variability of blocking frequency was shown to be quite large, demonstrating how difficult it can be to achieve a representative sample with only a few years of data. The predictive skill of the probabilistic forecasts of actual blocking was substantially smaller than its perfect-model skill, whereby a member of the ensemble was used as a synthetic verification. This indicates that there is still tremendous potential for improvement in blocking forecasts. However, it is also likely that the perfect-model results present a somewhat overoptimistic estimate of the upper range of forecast skill. The GEFS system and most other ensemble systems are underdispersive, and as such, the members of the ensemble unduly resemble each other, inflating the perfect-model skill estimates. It was also found that block onset and cessation were forecast somewhat less well than block maintenance, and there was substantial variability of blocking skill between half-decadal periods. Finally, the reliability of probabilistic blocking forecasts degraded with increasing lead time, and as expected, blocking forecasts became progressively less sharp (i.e., forecast probabilities were less often 0.0 and 1.0 and more often resembled the model climatology).
Forecasts of strong MJOs propagated too slowly, especially the component associated with outgoing longwave radiation (OLR; i.e., convection). Deep tropical convection appeared to have other systematic biases in the GEFS; in general, there was too much tropical precipitation forecast in the Indian and Pacific Oceans, especially for the shorter forecast leads. The ensemble predictions were biased and/or underdispersive, manifested in U-shaped rank histograms of MJO indices. Forecasts of the magnitude of the MJO's leading EOFs were especially U shaped. Bivariate MJO correlation skill was found to be larger for the wind component than for the OLR component, and skill was larger for the higher-amplitude MJO events. Skill varied significantly between half-decadal periods, with the period 1985–94 and 2010–12 exhibiting lower MJO skill than the 1995–2009 period. Probabilistic skill of the MJO forecast was modest, and skill was larger when measured relative to climatology than when measured relative to a lagged persistence forecast. Finally, for longer-lead forecasts, the GEFS demonstrated little ability to replicate the changes in blocking frequency due to a strong MJO that were noted in analyzed data.
This paper has discussed forecast skill without providing analysis of the potential forecast systematic errors that may lead to deficiencies in blocking, the MJO, and their interrelationships. This much more challenging work is left as future research. We do hope that we have laid out a first step, demonstrating the predictability and forecastability of these phenomena using the newly created GEFS reforecast dataset.
Acknowledgments
The reforecast dataset used here was created under a 2010 Department of Energy Advanced Leadership Computing Challenge supercomputer grant. This manuscript was partly supported by a NOAA USWRP grant on methods for estimating model uncertainty. We thank John Cortinas, the NOAA program manager, for his assistance and guidance on USWRP funding.
REFERENCES
Barnes, E. A., J. Slingo, and T. Woolings, 2012: A methodology for the comparison of blocking climatologies across indices, models, and climate scenarios. Climate Dyn., 38, 2467–2481, doi:10.1007/s00382-011-1243-6.
Bougeault, P., and Coauthors, 2010: The THORPEX Interactive Grand Global Ensemble. Bull. Amer. Meteor. Soc., 91, 1059–1072.
Brown, J. R., S. B. Power, F. P. Delage, R. A. Colman, A. F. Moise, and B. F. Murphy, 2011: Evaluation of the South Pacific convergence zone in IPCC AR4 climate model simulations of the twentieth century. J. Climate, 24, 1565–1582.
Buizza, R., 1997: Potential forecast skill of ensemble prediction and spread and skill distributions of the ECMWF ensemble prediction system. Mon. Wea. Rev., 125, 99–119.
Cassou, C., 2008: Intraseasonal interaction between the Madden–Julian Oscillation and the North Atlantic Oscillation. Nature, 455, 523–527.
Colucci, S. J., and T. L. Alberta, 1996: Planetary-scale climatology of explosive cyclogenesis and blocking. Mon. Wea. Rev., 124, 2509–2520.
Cressman, G. P., 1959: An operational objective analysis system. Mon. Wea. Rev., 87, 367–374.
Crueger, T., B. Stevens, and R. Brokopf, 2013: The Madden–Julian oscillation in ECHAM6 and the introduction of an objective MJO metric. J. Climate, 26, 3241–3257.
Drosdowsky, W., and L. E. Chambers, 2001: Near-global sea surface temperature anomalies as predictors of Australian seasonal rainfall. J. Climate, 14, 1677–1687.
Ferranti, L., T. N. Palmer, F. Molteni, and E. Klinker, 1990: Tropical–extratropical interaction associated with the 30–60 day oscillation and its impact on medium and extended range prediction. J. Atmos. Sci., 47, 2177–2199.
Folland, C. K., J. A. Renwick, M. J. Salinger, and A. B. Mullan, 2002: Relative influences of the interdecadal Pacific Oscillation and ENSO in the South Pacific convergence zone. Geophys. Res. Lett.,29,doi:10.1029/2001GL014201.
Fu, X., J.-Y. Lee, P.-C. Hsu, H. Taniguchi, B. Wang, W. Wang, and S. Weaver, 2013: Multi-model MJO forecasting during DYNAMO/CINDY period. Climate Dyn., 41, 1067–1081.
Gelaro, R., R. H. Langland, S. Pellerin, and R. Todling, 2010: The THORPEX observation impact intercomparison experiment. Mon. Wea. Rev., 138, 4009–4025.
Gottschalck, J., and Coauthors, 2010: A framework for assessing operational Madden–Julian Oscillation forecasts. Bull. Amer. Meteor. Soc., 91, 1247–1258.
Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129, 550–560.
Hamill, T. M., and J. Juras, 2006: Measuring forecast skill: Is it real skill or is it the varying climatology? Quart. J. Roy. Meteor. Soc., 132, 2905–2923.
Hamill, T. M., J. S. Whitaker, D. T. Kleist, M. Fiorino, and S. G. Benjamin, 2011: Predictions of 2010's tropical cyclones using the GFS and ensemble-based data assimilation methods. Mon. Wea. Rev., 139, 3243–3247.
Hamill, T. M., G. T. Bates, J. S. Whitaker, D. R. Murray, M. Fiorino, T. J. Galarneau Jr., Y. Zhu, and W. Lapenta, 2013: NOAA's second-generation global medium-range ensemble reforecast dataset. Bull. Amer. Meteor. Soc., 94, 1553–1565.
Hendon, H. H., B. Liebmann, M. Newman, and J. D. Glick, 2000: Medium-range forecast errors associated with active episodes of the Madden–Julian Oscillation. Mon. Wea. Rev., 128, 69–86.
Huffman, G. J., R. F. Adler, M. Morrissey, D. T. Bolvin, S. Curtis, R. Joyce, B. McGavock, and J. Susskind, 2001: Global precipitation at one-degree daily resolution from multisatellite observations. J. Hydrometeor., 2, 36–50.
Jia, X., C. Li, N. Zhou, and J. Ling, 2010: The MJO in an AGCM with three different cumulus parameterization schemes. Dyn. Atmos. Oceans, 49, 141–163.
Jones, C., D. E. Waliser, K. M. Lau, and W. Stern, 2004: The Madden–Julian oscillation and its impact on Northern Hemisphere weather predictability. Mon. Wea. Rev., 132, 1462–1471.
Jung, T., and Coauthors, 2012: High-resolution global climate simulations with the ECMWF model in project Athena: Experimental design, model climate, and seasonal forecast skill. J. Climate, 25, 3155–3172.
Kalnay, E., and Coauthors, 1996: The NCEP/NCAR 40-Year Reanalysis Project. Bull. Amer. Meteor. Soc., 77, 437–471.
Kang, I.-S., and H.-M. Kim, 2010: Assessment of MJO predictability for boreal winter with various statistical and dynamical models. J. Climate, 23, 2368–2378.
Kiladis, G. N., and K. M. Weickmann, 1992: Circulation anomalies associated with tropical convection during northern winter. Mon. Wea. Rev., 120, 1900–1923.
Kim, D., and Coauthors, 2009: Application of MJO simulation diagnostics to climate models. J. Climate, 22, 6413–6436.
Kleist, D. T., D. F. Parrish, J. C. Derber, R. Treadon, W.-S. Wu, and S. Lord, 2009: Introduction of the GSI into the NCEP global data assimilation system. Wea. Forecasting, 24, 1691–1705.
Knutson, T. R., and K. M. Weickmann, 1987: 30–60 day atmospheric oscillations: Composite life cycles of convection and circulation anomalies. Mon. Wea. Rev., 115, 1407–1436.
L'Heureux, M. L., and R. W. Higgins, 2008: Boreal winter links between the Madden–Julian oscillation and the Arctic Oscillation. J. Climate, 21, 3040–3050.
Liebmann, B., and D. L. Hartmann, 1984: An observational study of tropical–midlatitude interaction on intraseasonal timescales during winter. J. Atmos. Sci., 41, 3333–3350.
Lin, H., G. Brunet, and J. Derome, 2008: Forecast skill of the Madden–Julian oscillation in two Canadian atmospheric models. Mon. Wea. Rev., 136, 4130–4149.
Lin, H., G. Brunet, and J. Derome, 2009: An observed connection between the North Atlantic Oscillation and the Madden–Julian oscillation. J. Climate, 22, 364–380.
Madden, R. A., and P. R. Julian, 1971: Detection of a 40–50 day oscillation in the zonal wind in the tropical Pacific. J. Atmos. Sci., 28, 702–708.
Maloney, E. D., and D. L. Hartmann, 2000a: Modulation of the eastern North Pacific hurricanes by the Madden–Julian oscillation. J. Climate, 13, 1451–1460.
Maloney, E. D., and D. L. Hartmann, 2000b: Modulation of hurricane activity in the Gulf of Mexico by the Madden-Julian Oscillation. Science, 287, 2002–2004.
Matsueda, M., and H. Endo, 2011: Verification of medium-range MJO forecasts with TIGGE. Geophys. Res. Lett., 38, L11801, doi:10.1029/2011GL047480.
Mauritsen, T., and E. Källén, 2004: Blocking prediction in an ensemble forecasting system. Tellus, 56A, 218–228.
Palmer, T. N., 2012: Towards the probabilistic earth-system simulator: A vision for the future of climate and weather prediction. Quart. J. Roy. Meteor. Soc., 138, 841–861, doi:10.1002/qj.1923.
Pelly, J. L., and B. J. Hoskins, 2003a: How well does the ECMWF ensemble prediction system predict blocking? Quart. J. Roy. Meteor. Soc., 129, 1683–1702.
Pelly, J. L., and B. J. Hoskins, 2003b: A new perspective on blocking. J. Atmos. Sci., 60, 743–755.
Riddle, E. E., M. B. Stoner, N. C. Johnson, M. L. L'Heureux, D. C. Collins, and S. B. Feldstein, 2013: The impact of the MJO on clusters of wintertime circulation anomalies over the North American Region. Climate Dyn., 40, 1749–1766, doi:10.1007/s00382-102-1493-y.
Saha, S., and Coauthors, 2006: The NCEP Climate Forecast System. J. Climate, 19, 3483–3517.
Saha, S., and Coauthors, 2010: The NCEP Climate Forecast System Reanalysis. Bull. Amer. Meteor. Soc., 91, 1015–1057.
Seo, K.-H., W. Wang, J. Gottschalck, Q. Zhang, J.-K. Schemm, W. R. Higgins, and A. Kumar, 2009: Evaluation of MJO forecast skill from several statistical and dynamical forecast models. J. Climate, 22, 2372–2388.
Straub, K. H., 2013: MJO initiation in the real-time multivariate MJO index. J. Climate, 26, 1130–1151.
Tibaldi, S., and F. Molteni, 1990: On the operational predictability of blocking. Tellus, 42A, 343–365.
Trenberth, K. E., 1997: The definition of El Niño. Bull. Amer. Meteor. Soc., 78, 2771–2777.
Vitart, F., and F. Molteni, 2010: Simulation of the Madden–Julian Oscillation and its teleconnections in the ECMWF forecast system. Quart. J. Roy. Meteor. Soc., 136, 842–855.
Waliser, D., and Coauthors, 2009: MJO simulation diagnostics. J. Climate, 22, 3006–3030.
Wang, W., S. Saha, H.-L. Pan, S. Nadiga, and G. White, 2005: Simulation of ENSO in the new NCEP Coupled Forecast System Model (CFS03). Mon. Wea. Rev., 133, 1574–1593.
Wang, W., M.-P. Hung, S. J. Weaver, A. Kumar, and X. Fu, 2013: MJO prediction in the NCEP Climate Forecast System version 2. Climate Dyn., doi:10.1007/s00382-013-1806-9, in press.
Watson, J. S., and S. J. Colucci, 2002: Evaluation of ensemble predictions of blocking in the NCEP global spectral model. Mon. Wea. Rev., 130, 3008–3021.
Weaver, S. J., W. Wang, M. Chen, and A. Kumar, 2011: Representation of MJO variability in the NCEP climate forecast system. J. Climate, 24, 4676–4694.
Wei, M., Z. Toth, R. Wobus, and Y. Zhu, 2008: Initial perturbations based on the ensemble transform (ET) technique in the NCEP global operational forecast system. Tellus, 60A, 62–79.
Weickmann, K., and E. Berry, 2009: The tropical Madden–Julian oscillation and the global wind oscillation. Mon. Wea. Rev.,137, 1601–1614.
Wheeler, M. C., and H. H. Hendon, 2004: An all-season real-time multivariate MJO index: Development of an index for monitoring and prediction. Mon. Wea. Rev., 132, 1917–1932.
Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. 2nd ed. Academic Press, 627 pp.
Yasunari, T., 1979: Cloudiness fluctuations associated with the Northern Hemisphere summer monsoon. J. Meteor. Soc. Japan, 57, 225–229.
Predictability here refers to an intrinsic property of the atmospheric process. It measures the time scale at which two initially similar but not identical perturbed initial conditions subject to the actual atmosphere's dynamics will become as different as random draws of atmospheric states. While numerical models are often used to estimate the unknown predictability, the predictability is not a forecast property, but a property of the atmosphere. Forecastability, in contrast, indicates the ability of the forecast model to provide guidance that the user will judge to have some value. Forecastability may be evaluated with many different metrics.
