Realistic simulations of the Madden–Julian oscillation (MJO) by global climate models (GCMs) remain a great challenge. To evaluate GCM simulations of the MJO, the U.S. CLIVAR MJO Working Group developed a standardized set of diagnostics, providing a comprehensive assessment of statistical properties of the MJO. Here, a suite of complementary diagnostics has been developed that provides discrimination and assessment of MJO simulations based on the perception that the MJO propagation has characteristic dynamic and thermodynamic structures. The new dynamics-oriented diagnostics help to evaluate whether a model produces eastward-propagating MJOs for the right reasons. The diagnostics include 1) the horizontal structure of boundary layer moisture convergence (BLMC) that moistens the lower troposphere to the east of a convection center, 2) the preluding eastward propagation of BLMC that leads the propagation of MJO precipitation by about 5 days, 3) the horizontal structure of 850-hPa zonal wind and its equatorial asymmetry (Kelvin easterly versus Rossby westerly intensity), 4) the equatorial vertical–longitudinal structure of the equivalent potential temperature and convective instability index that reflects the premoistening and predestabilization processes, 5) the equatorial vertical–longitudinal distribution of diabatic heating that reflects the multicloud structure of the MJO, 6) the upper-level divergence that reflects the influence of stratiform cloud heating, and 7) the MJO available potential energy generation that reflects the amplification and propagation of an MJO. The models that simulate better three-dimensional dynamic and thermodynamic structures of MJOs generally reproduce better eastward propagations. This evaluation identifies a number of shortcomings in representing dynamical and heating processes relevant to the MJO simulation and reveals potential sources of the shortcomings.
Tropical atmospheric motion exhibits a significant energy peak on a broad range of 2–10 weeks, which is often referred to as tropical intraseasonal variability. The Madden–Julian oscillation (MJO) is the dominant mode of tropical intraseasonal variability that bridges weather and climate variations (Zhang 2013) and is a major source of global predictability on the subseasonal time scale (Waliser et al. 2012). Notable progress has been made in steadily improving MJO simulations in global climate models (GCMs), but realistic simulation of the MJO in many current GCMs remains a great challenge (Jiang et al. 2015; Ahn et al. 2017).
Statistically, the MJO is defined in the cross spectrum of the wavenumber–frequency domain of outgoing longwave radiation (OLR) and 850-hPa zonal wind (U850) as the spectral components within zonal wavenumbers 1–3 and periods of 30–80 days (Wheeler and Kiladis 1999; Waliser et al. 2012). To evaluate and track GCMs’ simulations of the MJO and identify their major problems, the U.S. Climate Variability and Predictability (CLIVAR) MJO Working Group designed a suite of diagnostics (Waliser et al. 2009). Because of the strong seasonality of the tropical intraseasonal variability, the diagnostics were applied separately to boreal winter from November to April and boreal summer from May to October. The main diagnostic quantities included (i) seasonal variations of the mean circulation (supplementary metric) and intraseasonal variance and (ii) properties of the MJO. The properties of the boreal winter MJO characteristics were diagnosed by 1) lag–longitude correlation analysis, 2) single-variable empirical orthogonal function (EOF) analysis, 3) single-variable frequency–wavenumber spectra, 4) single-variable frequency–wavenumber diagrams (Wheeler and Kiladis 1999), 5) cross spectrum in the wavenumber–frequency domain (Hendon and Wheeler 2008), 6) multivariate EOF analysis, and 7) MJO life cycle composites in the horizontal domain (20°S–20°N, 0°–360°) and in the vertical domain (0°–360°, 0-hPa surface). The variables examined primarily included OLR, precipitation, U850, 200-hPa zonal wind (U200), and surface zonal wind except in the life cycle composites, where additional variables were used: meridional winds, temperature, humidity, vertical velocity, sea level pressure (SLP), and 200-hPa streamfunction, as well as sea surface temperature (SST) and surface latent and sensible heat fluxes. These diagnostics provide a comprehensive assessment of the simulated MJO properties, but tend to reflect the statistical behavior of the MJO. In addition, for practical applications, one needs to decide which subset of the diagnostics and variables will be chosen because some of them are redundant. Furthermore, some diagnostics, for example the structures of the two multivariate EOF modes, are useful, but it is difficult to develop objective measures to quantify the fidelity with which models simulate the structures of EOF modes; the lead–lag correlation between the two leading EOFs reflects propagation, but does not represent simulated propagation skill well (Sperber and Kim 2012).
From a dynamical standpoint, the MJO can be defined as a planetary-scale, tropical circulation system, coupled with a multiscale convective complex, moving eastward slowly (~5 m s−1) over the warm pool of the Indo-Pacific oceans with a rearward-tilted baroclinic vertical structure and a coupled Kelvin–Rossby wave (horizontal) structure (Wang and Chen 2017). Based on this perception, the large-scale dynamic and thermodynamic structures of the MJO system should be taken into account in the diagnostics for assessing the fidelity of GCM simulations of the MJO.
Observations have shown that anomalous surface low pressure and boundary layer (BL) moisture convergence (BLMC) lead the major convective centers (Madden and Julian 1972; Wang 1988a; Hendon and Salby 1994; Salby et al. 1994; Jones and Weare 1996; Maloney and Hartmann 1998; Sperber 2003; Kiladis et al. 2005; Zhang 2005; Tian et al. 2006). To the east of the MJO convective center exist gradual deepening of the moist BL (Johnson et al. 1999; Kemball-Cook and Weare 2001; Tian et al. 2006), increasing convective instability (Hsu and Li 2012), poleward winds in the lower free troposphere and associated positive moisture advection (Kim et al. 2014; Wolding and Maloney 2015), and a transition from shallow cumulus, congestus clouds to deep convection and anvil stratiform clouds (Kikuchi and Takayabu 2004; Katsumata et al. 2009; Virts and Wallace 2010; Del Genio et al. 2012; Johnson et al. 2015). These structural features may well be critical to the slow eastward propagation of the MJO.
In theory, the MJO structure is a result of the three-way interaction among convective heating, moisture, and large-scale (wave and boundary layer) dynamics (Wang et al. 2016). Using a general theoretical model for the MJO, Wang and Chen (2017) have shown that different cumulus parameterization schemes or different parameters in the Betts–Miller scheme (e.g., the convective adjustment time scale) can produce different MJO structural asymmetries, especially the relative intensity of the low-level equatorial Rossby westerlies versus Kelvin easterlies (the R–K intensity ratio). They have also demonstrated that a large R–K intensity ratio of 2 [similar to the Gill (1980) pattern] corresponds to a nonpropagating MJO mode, a ratio of 1.0 (similar to the observed MJO) produces a slow eastward propagation speed of 5 m s−1, and a ratio of 0.6 corresponds to a fast propagation of 15 m s−1. This result suggests that a relative strong Kelvin easterly component favors eastward propagation, which is consistent with the previous aquaplanet GCM experiment results of Kang et al. (2013). Using a simplified Bretherton parameterization scheme, Adames and Kim (2016) have suggested a similar relationship between structural asymmetry and propagation speed.
To explore why the GCMs have a diversity of performances in the MJO simulation, Wang and Lee (2017, hereinafter WL17) diagnosed 24 GCM simulations and found that the models that simulate better structural asymmetry have better performance in simulations of eastward propagation of the MJO. A variety of structural asymmetries seen from the 24 models studied may arise from different representations of the convective heating. The models’ capacity in capturing correct the correct dynamic and thermodynamic structure of the MJO may suggest the models’ capability in adequate representations of the convective heating, moisture feedback, wave feedback, convective mixing, low-cloud and stratiform cloud feedback, and BL parameterization, among other processes. To help identify models’ deficiencies, it is crucial to assess the performance in simulation of the dynamical and thermodynamic structures and the associated diabatic heating and energy generation processes.
Motivated by the aforementioned observations, theoretical findings, and GCM simulation results, in this study, we aim to advance a suite of dynamics-oriented diagnostic metrics for evaluation of GCM simulations of the MJO. Different from the diagnosis of statistical properties, the dynamics-oriented diagnostics are based on the perception that the MJO is a dynamic system with characteristic dynamic and thermodynamic structures that are intimately related to its propagation and instability. The diagnostics will be built based on the observed rudimentary features of the MJO and our theoretical understanding of the essential MJO dynamics with specific attention paid to the processes associated with MJO propagation and amplification/decay. The proposed metrics for each diagnostic variable are intended to be physically intuitive, statistically robust, as well as easy to compute in order to quantitatively measure the GCMs’ skill.
The dynamics-oriented metrics are different from the existing “process oriented” metrics of the MJO (e.g., Kim et al. 2014). The latter focus on physical processes related to precipitation (e.g., its interaction with ambient moisture) at gridpoint scales and moisture perturbations or normalized gross moist stability (e.g., Benedict et al. 2014), whereas the dynamics-oriented metrics focus on large-scale dynamics and thermodynamic structures of the MJO. The two are complementary to each other. The process-oriented diagnostics intend to explain the failure of a model to produce the observed statistical signals of MJOs that can be attributed to deficiencies in model parameterization schemes, especially that for cumulus convection. The diagnostics introduced in the present study help reveal whether a model produces the statistical eastward propagation of the MJO for the right dynamical reason, and identify models’ shortcomings in representing the physical processes relevant to the MJO’s structure, propagation, and amplification, as well as the potential sources of the shortcomings.
2. Data and method
We use Global Precipitation Climatology Project (GPCP) daily precipitation data (Huffman and Bolvin 2013) for the period of 1997–2014 (18 yr). For the horizontal and vertical winds, temperature, diabatic heating, and specific humidity, the ERA-Interim reanalysis dataset with the 1.5° latitude × 1.5° longitude horizontal resolutions (Dee et al. 2011) are utilized. For all analyses in the present study, intraseasonal (20–70 day) bandpass-filtered anomalies during boreal winter (from November to April) are analyzed. The data are interpolated and averaged onto 2.5° × 2.5° grids in order to match the model output. Here, we define the observations to be GPCP precipitation and ERA-Interim combined.
In the present study, we analyze 24 separate 20-yr simulations from 22 GCMs (Table 1) that participated in the Vertical Structure and Diabatic Processes of the MJO: A Global Model Evaluation Project. To characterize, compare, and evaluate the heating, moistening, and momentum mixing processes associated with the MJO that are produced by current GCMs, this joint research activity was launched by the WCRP–World Weather Research Programme (WWRP)/THORPEX MJO Task Force and the Year of Tropical Convection (YOTC) and GEWEX Atmosphere System Study (GASS) (http://www.cgd.ucar.edu/projects/yotc/mjo/vertical.html) (Petch et al. 2011; Klingaman et al. 2015). Included is a 20-yr simulation component that aims at characterizing the models’ intrinsic capability to represent the MJO variability and explores key processes responsible for high quality representation of the MJO [Jiang et al. (2015); see also Klingaman et al. (2015)—the synthesis paper]. The simulations were integrated for 20 yr and provided 6-hourly data with 2.5° × 2.5° horizontal resolution and 22 vertical pressure levels. The detailed descriptions of the project and models can be found in Jiang et al. (2015). We use this dataset and apply the derived metrics to evaluate the models’ performance and discuss their potential deficiencies.
The diagnostics are developed by regressing 20–70-day filtered anomalies of all variables against the precipitation anomalies at the equatorial eastern Indian Ocean (EIO; 10°S–10°N, 80°–100°E). This location is chosen because during the boreal winter the intertropical convergence zone (ITCZ) is closest to the equator in the EIO and the mean state tends to be more equatorially symmetric; the corresponding MJO precipitation and circulation are also more symmetric about the equator, and thus the results are more suitable for the study of the structure of the intrinsic MJO mode with less influence of the mean flow.
The MJO mode in reality is affected by the mean climatology. During January–March, the MJO convection after passing through the equatorial EIO tends to move along the Australian monsoon trough and the South Pacific convergence zone and the MJO’s structure and behavior are more complicated. Similar and even more significant basic state modulation of the MJO mode occurs during boreal summer when the MJO’s eastward propagation weakens and pronounced northward propagation prevails in monsoon regions; the structure and life cycle are more complex (e.g., Wang and Rui 1990; Wang and Xie 1997; Kemball-Cook and Wang 2001; Jiang et al. 2004; Zhang 2005; Waliser 2006; Kikuchi et al. 2012; Chu et al. 2017; Neena et al. 2017). For simplicity and clarity, the present study focuses on the MJO during boreal winter.
All dynamics-oriented diagnostics proposed in the present study are measured by the pattern correlation coefficient (PCC) between the observations and model simulations. A conventional method is to directly compute the PCC between the observed and model simulated lag-correlation maps in the Asian–Pacific domain (e.g., Jiang et al. 2015). With that approach, even the models that have no eastward propagation would still have a PCC higher than 0.5. This is because the models may have a local stationary oscillation in the EIO (there are many such cases in the observations as well as in the models). In the lag-correlation map, the stationary oscillation in the EIO is concentrated in the 85°–95°E region and from −20 to +20 days. Unlike Jiang et al. (2015) and WL17, the longitude range between 85° and 95°E, where the correlation (or regression) coefficients represent local stationary oscillation, is excluded in the calculation of PCC in the time–longitude domain to better depict propagation characteristics. The exclusion of the local stationary oscillation at the reference point reduces the poor models’ PCCs, while the excellent models’ PCC remain unchanged, so the models’ PCC scores spread in a larger range than with the inclusion of the local oscillation.
The generation of MJO available potential energy (APE) is determined by the covariance between the diabatic heating Q and temperature T perturbations (Wang and Li 1994). Before calculation of the regressed field of MJO APE, we computed daily 3D fields of QT during the 20-yr period. Then, a 20–70-day bandpass filter was applied (Q′T′). The lag-zero regression pattern of MJO APE is determined by regression of Q′T′ against the 20–70-day bandpass-filtered precipitation averaged over the equatorial EIO. The regression amplitudes are scaled to a fixed 3 mm day−1 precipitation rate for comparison.
3. Performance measures of the simulated MJO
Among several basic features of the MJO, the eastward propagation is its most essential feature (Madden and Julian 1972). It is the slow eastward propagation that characterizes the life cycle of the MJO and produces its prominent 30–60-day time scale. Thus, the eastward propagation is used to represent the primary fidelity of the GCM-simulated MJO compared to the observations.
A straightforward way of depicting MJO eastward propagation is the lag–longitude correlation analysis of precipitation anomalies proposed by the MJO Working Group (Waliser et al. 2009). Here, we use the 20–70-day filtered GPCP daily precipitation and averaged precipitation data within the EIO (10°S–10°N, 80°–100°E) as a reference region to construct lag–longitude correlation diagrams. The reference region covers a relatively large area (20° × 20°) to emphasize the planetary scale of the MJO.
Figure 1a shows the observed lag–longitude correlation map. The observed MJO precipitation exhibits continuous eastward propagation from 50°E to 180° with an average speed of about 5 m s−1. Simulation performance can readily be measured by the PCC between the observed and simulated propagation diagrams on the time–longitude domain (50°E–180°, from day −20 to day 20) by exclusion of the longitude range between 85° and 95°E for the reasons discussed in section 2. Figure 1d shows the PCC skill of each model, which varies from 0.20 to 0.96. Based on the PCC values, we select the six best models, with an average PCC of 0.93 (hereinafter excellent models), and the six poorest models with an average PCC of 0.47 (hereinafter poor models). Figures 1b and 1c present the composite lag–longitude correlation diagrams for the excellent and poor models, respectively. Obviously, the excellent models simulate realistic eastward propagation, while the poor models fail in a statistical sense. In general, higher PCC values correspond to more systematic eastward propagation of both the wet and dry anomalies and better propagation speeds over the equatorial Indo-Pacific warm pool regions.
Note that the PCC value is an overall assessment of the propagation feature; but it cannot distinguish between quality in simulated propagation speed and quality in propagation distance and continuity. Statistically, the eastward propagation can also be reflected in the single-variable frequency–wavenumber diagram or cross spectrum in the wavenumber–frequency domain as proposed by the MJO Working Group, where the ratio of the spectral power for the eastward- and westward-propagating components (the E/W ratio) on the MJO time and spatial scales was used to measure the eastward propagation skill (Kim et al. 2009). This spectral measure (the E/W ratio) has been shown to correlate very well with the PCC skill score in the lag–longitude diagram (CC = 0.78 for 27 GCM simulations) (Jiang et al. 2015). Therefore, use of the lag–longitude correlation diagram is probably a simple, yet sufficient, way to measure the MJO propagation and performance skill.
It is worth noting that the poor models that do not produce the statistical eastward-propagating signals of the MJO may still produce infrequent individual MJO events (Ling et al. 2017). Any diagnostics based on statistical approaches (e.g., regression, composite) would not be able to characterize signals of these infrequently produced MJO events. This is a common limitation of all diagnostics that seek statistical composite signals of the MJO. In future study, we plan to apply the dynamical diagnostics to be introduced in section 4 to individual MJO events simulated by GCMs regardless of whether or not they can produce statistical eastward-propagating signals of the MJO.
4. Diagnostic metrics
a. Structure and propagation of boundary layer moisture convergence
Madden and Julian (1972) noted an important feature of the MJO; that is, the low SLP anomaly is located to the east of the major convective center. This feature has been little discussed in the MJO literature, although the BL convergence in the anomalous low has been emphasized (Wang 1988b; Hendon and Salby 1994; Maloney and Hartmann 1998). The anomalous low implies not only BL convergence but also the strength of the Kelvin wave easterly and associated anomalous Walker cell to the east of the MJO convection, which may play a critical role in the MJO’s propagation. In the three-way interaction theoretical model, Wang et al. (2016) used three different simplified cumulus parameterization schemes to represent the moisture feedback. In that model, no matter which convective parameterization scheme is used, without the BL effect, an initial Gill-like disturbance will be decoupled with the Kelvin waves moving eastward and the Rossby waves westward. It is the BLMC that couples the Kelvin and Rossby waves and convection together, and selects eastward propagation under different heating schemes. WL17 showed that the BLMC favors the MJO eastward propagation by generating lower-tropospheric heating and APE to the east of precipitation center. Given this observed feature and the insight gained from theoretical work, we recommend the BLMC be considered as one of the diagnostic targets.
Figure 2a shows the observed horizontal structure of the BLMC. A notable feature is the zonal asymmetry of the BLMC with regard to the MJO precipitation anomaly. The observed BLMC extends to about 60° longitudes east of the EIO (90°E) with a center located at 130°E. The zonal asymmetry in the BLMC can be better seen from the equatorial distribution of BLMC averaged between 5°S and 5°N (Fig. 2b). The observation and excellent model composite show a strong BLMC over the Maritime Continent and equatorial western Pacific between 120° and 160°E while the poor model composite shows little of the BLMC over the corresponding region. Figure 2c shows that the quality of simulated MJO eastward propagation is significantly linked to the simulated BLMC structure with a positive CC of 0.79.
In addition to the BLMC structure, the observed BLMC propagates eastward systematically from 50°E to 180° at a speed of about 5 m s−1 (Fig. 3a). More importantly, the propagation of the BLMC leads that of the precipitation by about 5 days. Since the BLMC leads the precipitation propagation, the BLMC should be considered to be an indicator of the MJO eastward propagation. The performance of GCMs in simulating BLMC propagation can be measured by the PCC score between the observations and simulations in the domain of (−20 to+20 day and 50°E–180°). Figure 3b shows a strong positive correlation (CC = 0.83) between the performance of MJO propagation performance and that of the BLMC propagation, indicating that the models simulating better BLMC propagation reproduce more realistic eastward propagation of the MJO.
b. Zonal asymmetry in the low-level circulation
It has been found that the observed MJO propagation is intrinsically linked to its low-level horizontal circulation structure (WL17). The observed 850-hPa circulation anomaly has a Kelvin wave component to the east of the heating and a symmetric Rossby wave component to the west of the heating (Fig. 4a). As pointed out by WL17, the observed MJO structure, while resembling a Gill (1980) pattern, has notable differences with the Gill pattern in the relative strength and zonal extent of the Rossby and Kelvin (R–K) wave components. The zonal extent ratio of the Kelvin easterly versus that of the Rossby westerly is 3.0 in the Gill pattern but only 2.1 in the observed MJO; the maximum Rossby westerly speed versus the maximum Kelvin easterly speed is 2.2 in the Gill pattern but only 0.8 in the observed MJO. The different structures likely arise from the nature of the heating: the heating is specified in the Gill model whereas it is interactive with the circulation of the MJO, although other factors, such as basic flows and uneven boundary conditions, may also affect the structure. The GCMs may produce different horizontal structures because their cumulus parameterization schemes may lead to different interactions between convective heating and equatorial wave dynamics. Thus, the horizontal structure of the MJO should be an indispensable target of the diagnostics.
To quantify the performance in reproducing the observed U850 structure (Fig. 4a), we measure the overall shape of U850 in the domain of 15°S–15°N, 50°–160°E using the PCC between the observed and simulated U850. Figure 4c shows a robust relationship between the simulated MJO eastward propagation and the U850 structure with a CC = 0.74 (p < 0.01).
The difference between the excellent and poor simulations lies mainly in the zonal asymmetries in the relative intensity and zonal extent of the equatorial Kelvin (wave) easterlies versus Rossby (wave) westerlies (Fig. 4b). The equatorial structure of U850 in the observations is reproduced well by the excellent models but not by the poor models (Fig. 4b). An alternative metric for depicting the U850 structural asymmetry is the ratio of maximum intensity between the equatorial Kelvin wave easterlies and the equatorial Rossby wave westerlies. Figure 4d shows that the U850 asymmetric index simulated in the various models is well correlated with the performance of the simulated MJO eastward propagation (CC = 0.67, p < 0.01), but the relationship seems to be nonlinear, so the linear CC is not as high as the two-dimensional U850 structure. The observed ratio of maximum Kelvin easterlies versus Rossby westerlies is 1.25, indicating stronger Kelvin easterly waves than Rossby westerly waves. The excellent models’ mean is 1.16, resembling the observed MJO structure, but the poor models’ mean is 0.50, which resembles the Gill pattern.
c. Vertical structure of the equivalent potential temperature and convective instability
In the observations, deepening of the lower-tropospheric moist layer occurs before the enhanced deep convective phase of the MJO (e.g., Bladé and Hartmann 1993; Johnson et al. 1999; Kemball-Cook and Weare 2001; Tian et al. 2006; Zhu et al. 2009). This is also demonstrated by the eastward shift of the maximum BL specific humidity with respect to the convective center and a rearward- (westward-) tilted moisture field in the lower troposphere (Sperber 2003; Tian et al. 2006). Hsu and Li (2012) have shown that the equivalent potential temperature (EPT) also exhibits the rearward-tilted structure in the lower troposphere, and the convective instability measured by the difference in the EPT between the BL and middle troposphere is enhanced before the arrival of major MJO convection. The vertical gradient of EPT can indicate convective instability only if the layer is lifted to become saturated. Jiang et al. (2015) examined the vertical structure of the specific humidity and found the simulated vertical structure to be very well correlated with the simulated eastward propagation. Building on all this previous work, the vertical EPT profile representing the moist thermodynamic structure of the MJO is considered to be a basic diagnostic for the MJO.
The observed moist thermodynamic feature is confirmed by the vertical profile of the EPT over the EIO (Fig. 5a). The maximum EPT is found at 500 hPa, overlaying the convective center. There is a salient rearward (westward and upward) tilt of the EPT in the lower troposphere below the 500-hPa level, reflecting the gradual deepening of the moist layer toward the west on the east side of the convective center. Also of interest is the forward (eastward) tilt of the EPT in the upper troposphere between 400 and 200 hPa. The forward tilt, while not discussed much in the literature, may be mainly attributed to the positive temperature anomaly to the east of the deep convection center (Fig. 10b in WL17) because in the upper troposphere the moisture contribution to the EPT generation is small. This feature is important for the MJO eddy APE generation, as will be discussed shortly. However, it remains unclear what causes the upper-level positive temperature anomaly to the east of the MJO deep convective region. Net radiative heating may be a candidate factor (Del Genio and Chen 2015; Johnson et al. 2015). Whether this is related to the vertically propagating Kelvin waves (Virts et al. 2010) or the baroclinic Kelvin wave signature to the east of the MJO heating needs to be further investigated.
The rearward tilt of the EPT in the lower troposphere in Fig. 5a suggests that the premoistening and predestabilization processes precede the MJO convective peaks. Following Hsu and Li (2012) and WL17, a convective instability index is defined by the 850-hPa EPT minus 400-hPa EPT averaged between 120° and 150°E, which measures the predestabilizing conditions to the east of the MJO convection. Selection of the longitudinal range is based on the zonal distributions of the convective instability parameter in the observations, excellent model composites, and poor model composites (Fig. 5b). The distinct zonal asymmetry is simulated well by the excellent models, but the poor models cannot capture the observed high convective instability between 120° and 150°E.
To quantify the characteristic EPT profile, we use the PCC between the observed and model-simulated vertical profiles of the EPT along the equator from 40°E to 180° and from 1000 to 200 hPa. As shown in Fig. 5c, the PCC score of the simulated vertical structure of EPT is positively correlated with the MJO propagation score (CC = 0.77, p < 0.01). Another metric measuring the predestabilization (i.e., the convective instability index) is also reasonably well correlated to the simulated MJO propagation skills in all 24 GCM simulations with a positive CC of 0.62.
d. Zonally asymmetric distribution of diabatic heating
Observations have shown a continuous (or stepwise) transition from the shallow cumulus and congestus clouds to deep convective clouds and stratiform clouds (Kikuchi and Takayabu 2004; Katsumata et al. 2009; Virts and Wallace 2010; Del Genio et al. 2012; Johnson et al. 2015). Associated with the transition of clouds, the diabatic heating of the MJO possesses a vertical rearward-tilted structure (Sperber 2003; Jiang et al. 2011). The lower-tropospheric heating has been speculated to be important to the MJO (Li et al. 2009; Zhang and Song 2009). The importance of the stratiform cloud heating and its interaction with the second baroclinic mode and moisture has also been found to support MJO growth and propagation (Mapes 2000; Kuang 2008; Fu and Wang 2009).
Diabatic heating is dominated by condensational heating in the tropics and thus is an indicator of the precipitating cloud distributions. The longitudinal distribution of diabatic heating along the equator may suggest how the GCMs capture the observed transition from shallow cumuli, congestus to deep convective and stratiform clouds. The GCM-simulated cloud distribution provides critical information about cumulus parameterization and the representation of cloud in the GCMs.
Figure 6a shows an anomalous diabatic heating distribution and anomalous Walker cell in the zonal–vertical equatorial plane. The heating was computed from the reanalysis data using the budget residual analysis of Yanai et al. (1973). The diabatic heating results here are in good agreement with those of Johnson et al. (2015), which were calculated using sounding data during the DYNAMO field experiments (Yoneyama et al. 2013). The observed lower-tropospheric heating is clearly leading the midtropospheric deep convective heating (Fig. 6a), suggesting the existence of shallow and congestus clouds; meanwhile, the upper-tropospheric heating between 500 and 300 hPa tends to expand westward from the deep convective center to 60°E, suggesting that stratiform clouds may follow deep convection. Associated with the diabatic heating field are two Walker-like east–west cells (Madden and Julian 1972). One is to the west and the other to the east of the major convection. We refer to the one located to the east as the front Walker cell (FWC), as it leads the MJO propagation. The ascending branch of the FWC is coupled with the deep IO convection; whereas the descending branch corresponds to reduced diabatic heating over the western Pacific. The FWC can be enhanced by the zonal heating gradients between the IO and western Pacific (G. Chen and B. Wang 2017, manuscript submitted to J. Climate). The excellent models simulate realistic vertical tilt of diabatic heating and FWC (Fig. 6b) while the poor models fail to reproduce them (Fig. 6c), as previously noted by Jiang et al. (2015) and WL17. In the poor models, the diabatic heating is narrowly trapped in the deep convective regions with a center slightly to the west of the precipitation center, showing little sign of the lower-tropospheric heating leading deep convection and the westward extension of the upper-tropospheric heating.
For a given MJO precipitation rate of 3 mm day−1, both excellent and poor GCMs tend to produce stronger low-level wind responses than the observations, especially the low-level equatorial westerly anomalies. The strong responses of the low-level circulations in the models seem to be related to the lower maximum heating and larger vertical heating gradients in the lower troposphere. The maximum heating center is located around 420 hPa in the reanalysis, while it is around 460 hPa in the excellent models and 500 hPa in the poor models. In addition, the vertical heating gradients below 600 hPa simulated in the excellent models are twice as large as those in the reanalysis; even in the poor models this vertical gradient is larger than the observed.
The model performance in capturing the vertical heating structure is measured by the PCC between the observed and simulated vertical distributions of the equatorial diabatic heating in the domain of 40°E–180°, 1000–200 hPa. As shown in Fig. 6d, the PCC score between the observed and simulated vertical profile of equatorial diabatic heating is highly indicative of the quality of the model-simulated propagation (CC = 0.86, p < 0.01). Thus, the quality of a GCM’s simulation of diabatic heating processes is an important metric for the evaluation of MJO simulation.
e. Zonal asymmetry in the upper-level divergence and diabatic heating
The upper-level divergence is a good indicator of the MJO’s eastward propagation (Adames and Wallace 2014). It also reflects the upper-level precipitation heating (deep convection and the stratiform clouds tailing it). The longitudinal distribution of the 200-hPa wind divergence along the equator may reveal how the GCMs capture the observed transition from deep convective to stratiform clouds and how the GCMs produce upper-level divergence sources that are important for the tropical–extratropical teleconnection.
Figure 7a shows the observed 200-hPa winds and associated divergence when the convective center is located at the EIO. The upper-level circulation is approximately out of phase with that at 850 hPa (Fig. 4a), consisting of a pair of anticyclonic Rossby gyres and associated equatorial easterlies to the west of the convection and a Kelvin wave high and associated equatorial westerlies to the east of the convective center. Note that the equatorial U200 exhibits a strong zonal asymmetry with the Rossby easterly anomalies being much stronger than the Kelvin westerly anomalies, which is opposite to the zonal asymmetry in the U850 results. The 200-hPa divergence center is located to the west of the convection center and extends westward to the western IO, which largely coincides with the 300-hPa diabatic heating. Thus, we speculate that the westward extension of the 200-hPa divergence is primarily attributed to the westward extension of the 300-hPa diabatic heating produced by the stratiform clouds. There is also 200-hPa convergence occurring over the South China Sea and Philippines and the equatorial western Pacific between 150°E and 180°. Associated with the convergence over the Philippines is the divergence over East Asia and the subtropical western Pacific, a sign of teleconnection with the midlatitudes. The aforementioned observed features are well simulated by the excellent models (Fig. 7b). But in the poor model, the divergence does not extend westward to the western IO and very weak convergence occurs to the east of the major convective center (Fig. 7c).
Similar to the low-level circulation field, we calculate the PCC of 200-hPa divergence between the observations and simulations (15°S–15°N, 50°–160°E) to quantify the models’ capability for capturing the upper-level circulation. Figure 7d shows that the models simulating better upper-level divergence provide better simulations of the MJO eastward propagation with a CC of 0.75, suggesting that the 200-hPa divergence field is a useful metric for gauging the GCMs’ performance. Similarly, we measure the PCC of 300-hPa diabatic heating over the same region (15°S–15°N, 50°–160°E). As shown in Fig. 7e, the models’ PCC skill in simulations of the 300-hPa diabatic heating is well correlated with the models’ performance in simulations of the MJO propagation (CC = 0.74). Thus, the 200-hPa divergence (or 300-hPa diabatic heating) can be used for evaluation of GCM simulations of the upper-level circulation.
f. MJO available potential energy generation
The observed MJO convective anomalies show amplification over the warm IO and equatorial western Pacific, decaying over the Maritime Continent and near the date line when approaching the cold tongue of the eastern Pacific (Madden and Julian 1972; Wang and Rui 1990). The SST largely controls the basic-state moist static energy (MSE) distribution and, thus, the MSE advection associated with MJO. The SST also regulates the intensity of deep convection, thereby affecting the amplitude of the MJO. Over the EIO the amplification and propagation of the MJO can be diagnosed by energetic analysis. The eddy APE is the energy source for MJO development. WL17 showed that the large difference in eddy APE generation between the excellent and poor models lies in its zonal asymmetry. Thus, we consider the vertical structures of eddy APE generation to be an indicator for the MJO simulation.
Figure 8a displays the rate of generation of the MJO APE along the equator from 40°E to 180°. The eddy APE is determined by the phase overlapping (covariance) of the diabatic heating and temperature anomalies (heating where air is anomalously warm and cooling where it is anomalously cold). Near the major convective center, the observed APE generation rate is large in the upper troposphere (above 550 hPa) because the diabatic heating reaches a maximum (Fig. 6a) and the air tends to be warmer than normal (Fig. 8a). Notably, the observed APE generation to the east of 90°E is significantly larger than that to the west of 90°E, especially in the lower troposphere. The zonal asymmetry in APE generation with respect to the MJO precipitation center is primarily due to (i) positive temperature anomalies occurring to the east of the major convection region (Fig. 8a) and (ii) the lower-tropospheric heating occurring to the east of the 90°E (Fig. 6a).
Note that the MJO APE generated in the major convective region mainly contributes to the amplification of the MJO. But more generation of APE to the east of the major convective center facilitates the eastward propagation of the MJO because the APE would be converted to the MJO kinetic energy during which the upward motion is promoted to the east of the MJO convection.
The PCC between the observed and simulated APE generation rates over the equatorial domain (40°E–180° and 1000–200 hPa) is used to measure this simulated asymmetry. The performance of the models in simulations of the APE generation has a significant positive correlation (CC = 0.84) with that of the model-simulated MJO propagation (Fig. 8b).
Despite the considerable progress that has been made in the last two decades, simulations of the MJO in GCMs still suffer from notable shortcomings. The CLIVAR MJO Working Group developed a standardized, comprehensive set of diagnostics for assessing models’ successes and shortcomings in the MJO simulations. These diagnostics tend to reflect the statistical behaviors of the MJO. The MJO is a dynamic system with characteristic dynamic and thermodynamic structures that are intimately related to its propagation and instability. Within this conceptual framework, an additional and complementary set of dynamics-oriented diagnostics is presented in this current study. This set of diagnostics aims to focus on the observed fundamental dynamical structures and thermodynamic features of the MJO deemed important to its propagation and amplification/decay. To quantitatively assess the GCMs’ skill at the MJO propagation and reasons for possible shortcomings, the metrics for various diagnostic fields are designed, aiming to be physically intuitive, statistically robust, and easy to compute.
In the present study, the MJO propagation is depicted by the lag-correlated precipitation anomalies along the equatorial channel with reference to the EIO convective center (Fig. 1). It is also desirable to assess a similar lag-correlation map with reference to the Maritime Continent and western Pacific convection (WL17). To capture and quantify the fidelity of a GCM to replicate the observed elementary features of the MJO during boreal winter [November–April (NDJFMA)], the dynamics-oriented diagnostics include the seven components summarized in Table 2: 1) the horizontal structure of BLMC (Fig. 2) that moistens the lower troposphere to the east of convection center; 2) the preluding propagation of the BLMC (Fig. 3), which leads precipitation propagation by about 5 days; 3) the horizontal structure and zonal asymmetry in U850 (Fig. 4), which is characterized by the relative intensity of the Kelvin versus Rossby wave components that is intrinsically linked to the MJO propagation; this low-level horizontal circulation quality may be relevant to explaining the importance of the horizontal advective moistening for eastward propagation (Maloney 2009); 4) the equatorial vertical structure of the EPT (Fig. 5) or MSE, which reflects the observed premoistening process that gradually deepens the lower-tropospheric moist layer and increases convective instability; 5) the equatorial vertical–longitudinal distribution of diabatic heating and associated anomalous Walker cells (Fig. 6), which reflects well the multicloud structure of the MJO convective complex and a continuous transition from shallow cumulus–congestus clouds to deep convective-stratiform clouds; 6) the upper-level divergence and diabatic heating (Fig. 7) which is zonally asymmetric and a good indicator of the MJO deep convective and stratiform clouds; its longitudinal distribution may reveal how the GCMs simulate the eastward propagation and the sources for teleconnections to the midlatitude; and 7) the generation of MJO APE (Fig. 8), which reflects the amplification and propagation of the MJO, in particular the amplification over the IO and equatorial western Pacific warm pool and the decay over the Maritime Continent (Zhang and Ling 2017) and the central Pacific. The quality of the model-simulated seven diagnostic fields reflects very well the quality of the model-simulated eastward propagation of the MJO, with the PCCs ranging from 0.74 to 0.86.
To facilitate a quick assessment of individual model’s performance when simulating the seven diagnostic fields, we have calculated the averaged PCC skill scores for each group of diagnosed models (excellent, good, fair, and poor) as a reference for comparison. The results are shown in Table 3. By comparison of the performance from any individual model with the reference skills shown in Table 3, one can get a general idea about how good an individual model’s simulation quality is compared to the 24 model simulations.
Some of these metrics shown in Table 2 are well correlated with other metrics. For example, the correlation between the BLMC propagation skill and U850 structure skill is 0.67. But, we keep all of the metrics listed in Table 2 mainly based on the consideration of intrinsic dynamics. As discussed earlier, each diagnostic field is selected to represent a specific dynamical feature or process. This does not mean that all of the diagnostic fields are equally important. Use of relatively independent diagnostic fields may lead to a more concise set of the diagnostics. For instance, if the diabatic heating, propagation of the BLMC, and structure of the U850 are chosen as the metrics, their weighted combination (derived by multiregression),
can yield a PCC skill score of 0.94 when representing the MJO precipitation propagation.
These diagnostics help to identify the shortcomings of the GCMs in representing not only the dynamic structure and thermodynamic features but also the physical processes relevant to the MJO propagation and the potential sources of the shortcomings. For instance, a lack of premoistening and predestabilization, as well as the preluding BLMC propagation, may hint at problems in representing the BL convergence and its interaction with shallow-congestus clouds through lower-tropospheric convective mixing and low-cloud feedback. Deficiencies in the simulated thermodynamic structure, diabatic heating distribution, and upper-level divergence may be indicative of problems in the models’ representation of the formation of congestus and stratiform clouds. A deficient horizontal structure may be related to the cumulus parameterization or uncertain parameters within parameterization schemes (e.g., Kim and Seo 2018). On the other hand, a deficient horizontal circulation could result in erroneous horizontal moisture advection. The latter has been shown to be important for MJO propagation (Maloney 2009; Maloney et al. 2010; Sobel and Maloney 2012, 2013; Adames and Kim 2016; Jiang 2017).
By illustrating the dynamical structures associated with simulated MJO disturbances, the diagnostics introduced here help us to evaluate whether models produce the MJO (according to the basic statistical diagnostics) for the right reasons (according to the basic dynamic diagnostics). That said, the diagnostics were not designed to confirm or refute specific theories for the underlying macroscopic mechanisms for the very existence of the MJO and its properties that are repeatedly observed. For example, a number of studies argue for the role of radiative feedbacks in driving the MJO (e.g., Raymond 2001; Bony and Emanuel 2005; Andersen and Kuang 2012; Sobel and Maloney 2012, 2013; Arnold and Randall 2015; Adames and Kim 2016). The diagnostics introduced here do not include this possible aspect of the MJO. Also, this set of dynamics-oriented diagnostics is not exclusive of other factors that may impact the MJO simulations. For instance, correct simulation of seasonal-mean moisture is important for the MJO simulation (Kim et al. 2011; Benedict et al. 2014; Gonzalez and Jiang 2017; Jiang 2017), and correct simulation of the annual cycle of the mean state and the seasonal migration of the convective variance is also important. The boreal summer intraseasonal oscillation is more complex as a result of its more complicated interaction with the mean state, including stronger land–ocean–atmosphere interactions. Addressing the boreal summer version of the MJO to the same level as addressed here will likely require additional/different diagnostics. The MJO interaction with the underlying ocean, interaction with the extratropics, and other factors are not included in this study. Diagnostics required to assess these additional factors deserve to be developed.
This work is supported by the National Science Foundation (Climate Dynamics Division Award AGS-1540783), the NOAA/CVP (NA15OAR4310177), and the Global Research Laboratory (GRL) Program of the Korean Ministry of Education, Science and Technology (MEST, 2011-0021927), as well as IBS-R028-D1. This is SOEST Publication Number 10323, IPRC Publication Number 1313, and Earth System Modeling Center (ESMC) Publication Number 202. This is PMEL Contribution Number 4664. Eric Maloney was supported by the National Science Foundation under Grant AGS-1441916 and the National Oceanic and Atmospheric Administration under Grants NA15OAR4310098 and NA15OAR4310099. DEW’s contribution to this study was carried out on behalf of the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics 526 and Space Administration.