## 1. Introduction

The need for uncertainty information related to numerical weather forecasts has been long recognized and, over the past two decades, nearly all major weather services have implemented operational global ensemble prediction systems (e.g., Buizza et al. 2005). The value of ensemble prediction systems can be described by their ability to provide flow-dependent estimates of forecast uncertainty. Verification and diagnostics of ensemble performance focus on reliability and accuracy of the forecast of the whole PDF. Representativeness of ensemble spread in comparison to the forecast error growth is assessed by a variety of scores (e.g., Leutbecher and Palmer 2008). For example, ensemble reliability has been routinely measured by comparing the variance of the ensemble with the mean-squared error of the ensemble mean (e.g., Buizza et al. 2005; Hagedorn et al. 2012; Buizza 2014).

Many aspects of the ensemble forecasting remain challenging, in particular the generation of initial conditions. The European Centre for Medium-Range Weather Forecasts (ECMWF) operational ensemble (ENS), which is the dataset used in this paper, is based on initial perturbations generated by combining the ensemble of analyses produced by four-dimensional variational data assimilation (4DVAR) data assimilation (so-called EDA; Isaksen et al. 2010) and singular vectors (Buizza et al. 2008). An example of the zonally averaged global distribution of initial ensemble uncertainty in the ECMWF ENS is shown in Fig. 1a as the ensemble spread for a randomly chosen recent date. As seen in the figure, the initial perturbations are largest in the tropics close to the tropical tropopause. Their distribution closely resembles the distribution of the short-range forecast errors derived from the EDA ensemble and subsequent short-range forecasts (Žagar et al. 2013). A subsequent growth of initial uncertainties during the model evolution up to 7 days is presented in Figs. 1b–i. It shows that initially small uncertainties in the midlatitudes, represented by both the singular vectors and EDA components of the initial perturbations, grow much faster than the tropical uncertainties. After a few days, the dominant feature of the ensemble in the medium range is a large spread associated with the growth of baroclinic perturbations in the upper troposphere in the midlatitudes (e.g., Buizza and Palmer 1995).

Traditionally, diagnostics of ensemble performance has been presented for selected variables (temperature, wind components, and geopotential height) on standard pressure levels (usually 500 and 850 hPa). Furthermore, the performance has been analyzed separately for the midlatitudes (poleward of 20°N and 20°S) and the tropics (20°S–20°N). Typically, the midlatitude forecast errors (as measured by the comparison with analyses) tend to grow quasi exponentially in relation to baroclinically active scales where generation and propagation of the eddy kinetic energy take place (e.g., Lorenz 1982; Tribbia and Baumhefner 2004; Kuhl et al. 2007). On the contrary, the growth of ensemble uncertainties in the tropics has received little attention. Kuhl et al. (2007) used the perfect-model environment and the NCEP global forecast model to compare the midlatitude and tropical short-range forecast errors. They reported that tropical forecast errors grow linearly for all zonal wavenumbers and their magnitude is closely related to the magnitude of the analysis error. Magnusson et al. (2008) compared three different methodologies for initial perturbations for the ECMWF ensemble prediction system and showed that, after 2 days of forecasts, differences between the methods became small but the differences were largest in the tropics.

This paper introduces a new approach to the quantification of forecast uncertainty that considers both the midlatitudes and the tropics as well as wind and geopotential variables. The approach is based on the three-dimensional normal modes of the global atmosphere that include a number of vertical structure functions, each associated with a set of the horizontal Hough functions. The derivation relies on model levels and takes into account the surface pressure distribution and the topography. The derived modal view of forecast uncertainty represents the whole model depth and quantifies uncertainties associated with the balanced (quasigeostrophic or nearly rotational) and inertio-gravity (unbalanced) modes in terms of the ensemble variance along the three dimensions of the modal space: the zonal wavenumber, the meridional mode, and the vertical mode. The theory presented in section 2 shows that the new measure of forecast uncertainty is equivalent to the energy product for the 3D normal modes derived in Žagar et al. (2015). In the new formulation, ensemble reliability is defined in modal space and shown to provide information on motion types and scales characterized by a lack of variance.

The modal representation of forecast uncertainty is applied to the ECMWF ENS dataset which is described in section 3. Section 4 presents outputs of the modal space analysis; it shows how the predominantly unbalanced initial uncertainty of the ECMWF ENS evolves in time to become dominated by the balanced spread in planetary scales in midlatitudes. The main conclusions are stated in section 5.

## 2. Methodology

### a. Formulation of the normal-mode function representation in σ coordinates

The normal-mode function (NMF) expansion of 3D global data assumes that solutions to the adiabatic and inviscid linearized equations of atmospheric motions at time *t* are characterized by the separability of the vertical and horizontal dependences of the dependent variables describing the mass field and wind field. The presented representation of the ensemble spread is based on the 3D orthogonal NMFs that were derived by Kasahara and Puri (1981). Their derivation was based on the *σ* coordinate defined by *P* defined as *R* is the gas constant, and *g* is gravity. By assuming the separation of variables, global 3D motions are represented by horizontal structure equations with a form identical to that of the global shallow-water equations and the vertical structure equation. The two systems of equations are connected by the separation constant denoted *D*, which is known as the equivalent depth and it corresponds to the water depth for the horizontal motions (known also as the Laplace tidal equation without forcing). Solutions for the horizontal motions can be obtained analytically and have been studied by numerous authors (e.g., Hough 1898; Kasahara 1976; Phillips 1990; and references therein). They are best known as the Hough functions. For a detailed derivation, discussion of the Hough functions, and treatment of the zonally average state, the reader is referred to Kasahara (1976) and Kasahara (1978). The solutions to the vertical structure equation at *σ* levels with a realistic vertical temperature profile are obtained numerically applying boundary conditions of no mass transport through Earth’s surface and the model top

*j*th

*σ*level are represented by a series of the vertical structure functions

*t*,

*m*identifies the vertical mode, which varies from the external mode

*M*(in principle equal to the number of vertical levels—91 in our case). Functions

*J*sigma levels and solved by using the finite difference method, can be expressed as

*J*with the use of the orthogonality condition in Eq. (2). The result becomesEquations (1) and (4) are the vertical transform pair.

*m*are represented by a series of Hough harmonics

*K*, including zero for the zonal-mean state. The maximal number of meridional modes denoted by

*R*combines

*m*are characterized by the two indices for the zonal wavenumber

*k*and meridional mode

*n*.

*λ*from 0 to

*φ*from

### b. Ensemble variance in modal space

*P*(50 in our case). The ensemble spread in modal space is thus defined for each balanced and inertio-gravity mode and denoted

*i*and meridional index

*j*of the horizontal grids of the

*m*th shallow layer after the vertical transform in Eq. (4). The physical-space specific variance

*p*from the ensemble mean for wind components and geopotential height at location

*M*shallow-water layers by the vertical transform in Eq. (4). The vertical mode index

*m*takes values from

*p*from the ensemble mean asFirst, we multiply Eq. (12) by

*λ*from 0 to

*φ*from

*P*members we obtain Eqs. (9) and (11), respectively. Summing the resulting equations over the vertical and horizontal dimensions we obtain the total variance defined by Eqs. (8) and (10), respectively.

### c. Ensemble reliability in modal space

Reliability is an essential property of an ensemble-based, probabilistic system (e.g., Wilks 2011). A probabilistic system is reliable if, on average, events that are predicted with a probability *P* are also observed with a probability *P*, and this can be measured by metrics such as the Brier score (Brier 1950) or by assessing whether the ensemble variance is equal to the average mean-squared error of the ensemble mean. A reliable ensemble is characterized by an unbiased ensemble mean and provides samples of the future state of the atmosphere representative for the whole probability distribution of forecasts (Candille and Talagrand 2005). The quality of operational ensemble systems (with sizes of order 10–100) has been routinely assessed by comparing the ensemble standard deviation (ensemble spread) with the average error of the ensemble-mean forecast verified against operational analyses (Buizza et al. 2005; Hagedorn et al. 2012; Buizza 2014). In addition to the assumption of bias-free forecasts, for short lead-time ranges forecast errors are comparable and highly dependent on simulated analysis uncertainties. However, using observations instead of analyses to verify ensemble forecasts is difficult (Saetra et al. 2002). In particular, in our case of scale-dependent comparison of dynamical properties of the ensemble, the verification against the control analysis is the only approach possible.

In modal space, we represent ensemble variance along the three spatial dimensions (zonal wavenumbers and meridional and vertical modes) as well as along the two dimensions of mode types: inertio-gravity and balanced modes. Modal reliability expects that the ensemble variance agrees with the mean-squared error of the ensemble mean along any of these dimensions as well as integrated along any combination of these (as they are orthogonal).

*t*by

*P*ensemble members:The verifying analysis is denoted by

*t*and the control analysis valid at the same time,

*t*. The mean-squared error of the ensemble mean at forecast step

*t*, denoted

In a reliable ensemble, the true state of the atmosphere is on average included in the range spanned by the ensemble members, while in an underdispersive ensemble (i.e., in an ensemble with a spread that is on average smaller than the error of the ensemble mean), the true state can lay outside. Similarly, according to the modal diagnostic an underdispersive ensemble will be characterized by a lack of spread in one of directions of modal space or in the balanced and IG motions. Depending on the scales and motion types, the underdispersiveness can be associated with the tropical or midlatitude circulations.

## 3. ECMWF ensemble prediction dataset

At the time of writing (spring 2015), the ECMWF medium-range ensemble comprises 51 members: 1 unperturbed and without model error schemes and 50 perturbed ones. The ENS runs twice a day, with initial times at 0000 and 1200 UTC, up to 15 days; at 0000 UTC on Mondays and Thursdays the forecasts are extended to 32 days (Vitart et al. 2008). The forecasts are coupled to an ocean wave model (WAM; Janssen et al. 2005, 2013) and a dynamical ocean circulation model. The ocean circulation model is the Nucleus for European Ocean Modeling (NEMO; Mogensen et al. 2012a,b). A detailed description of how the ECMWF medium-range/monthly ensemble is generated is provided in the appendix.

### a. December 2014 ENS dataset for NMF representation

The analysis of the ENS system is performed for 1 month of data (December 2014) using the 0000 UTC run. For every analysis time, there are 15 forecast ranges available, with forecast lead times of 00, 12, 24, …, 168 h and model-level prognostic variable data. The range of operational ENS forecast is longer than analyzed (1 week), but for forecast ranges 7–10 days outputs are available with 24-h step and are not included in the study. As the ENS outputs are not stored in the ECMWF archiving system, a dedicated job during the study period extracted all 91 model-level data on the regular Gaussian N64 grid, which consists of 256 × 128 grid points equivalent to a grid spacing of about 150 km at the equator. Statistics are based on 50 perturbed forecasts. The initial state of the unperturbed forecast (denoted control analysis) is used for the computation of reliability. Results are shown as monthly means for 15 different time ranges.

The described modal representation of the spread provides properties of the ensemble spread across many scales that can be integrated vertically over the whole model depth. This is in contrast to the typical ensemble spread in physical space that is usually discussed for selected levels and variables, typically geopotential height and temperature in the middle troposphere. Thus we start our presentation by showing such curves and later compare them with the scale-dependent spread in modal space. Figure 1 shows zonally averaged zonal wind spread for the ENS forecast initialized at 0000 UTC 1 December 2014. Nine panels of this figure present the growth of spread in physical space during 7 days. It can be seen that the initial-time uncertainties are small (0.5–1 m s^{−1}) except in the tropics where the maximal spread around 2.5 m s^{−1} is located at the tropical tropopause. Figure 1a closely resembles the spatial distribution of the short-range forecast errors derived from the ECMWF 4DVAR ensemble method in recent years [see Fig. 2 in Žagar et al. (2013)]. The same figure for other dates in December 2014 as well as for more recent months looks very similar, confirming that the largest part of initial uncertainties is produced by the EDA method. The midlatitude wings of an increased spread in the upper troposphere grow with the forecast lead time and after 48 h the zonal wind spread in the midlatitude upper troposphere exceeds the spread at the tropical tropopause (Fig. 1d). Longer into the forecast the midlatitude spread grows larger, especially in the polar regions and in the lower troposphere. At forecast day 7, the zonally averaged midlatitude spread in the layer between 400 and 200 hPa is up to 15 m s^{−1} that is about 3 times greater than the maximal tropical spread in the upper tropical troposphere.

Figure 1 also shows initial uncertainties with large amplitudes in the upper stratosphere and mesosphere near the model top, especially in the tropics and subtropics. These uncertainties do not increase significantly in the tropics but do grow in the midlatitudes and they are primarily on large scales. As the SVs do not perturb stratospheric and mesospheric levels and planetary scales, these uncertainties come from EDA and they are a mixture of observation perturbations and evolved flow by the EDA cycling. There may be also a minor impact of vertical interpolation from L137 levels used in operational deterministic forecast and EDA to 91 levels used in ENS.

If Fig. 1 is averaged also latitudinally, the resulting curves for the globally averaged horizontal spread are displayed in Fig. 2. In general, the growth of the spread depends on the variable, region, and altitude. However, the globally averaged midlatitude spread maximizes in the upper troposphere at nearly all times (Fig. 2a) and for both wind components and geopotential, in agreement with the prevailing quasigeostrophic coupling between the forecast errors in the mass field and wind field. The secondary maximum is near the model top in the mesosphere. Figure 2d shows that as the forecast lead time increases, growth of the zonal wind spread becomes more inhomogeneous with a little growth in the midlatitude stratosphere. An important feature evident from Fig. 2 is that the normalized spread in 7-day range is maximal at 300 hPa in both the tropics and the midlatitudes, for both geopotential height and wind variables.

The vertically and horizontally averaged spread growth displayed in Fig. 3 shows that the growth rate is somewhat different for the wind and geopotential variables, with the latter showing clear exponential growth (Fig. 3c) known from many studies focusing on predictability (e.g., Lorenz 1982; Buizza 2010) while the former seem better characterized by a linear growth in the 7-day range. Growth curves for individual model levels have similar shape to that shown in Fig. 3 (not shown), suggesting that vertical averaging does not affect the shape of the curves. In particular, if the top 18 levels above 10 hPa or top 25 model levels above 30 hPa are excluded from the analysis because of a large spread in these levels, the normalized extratropical growth rate would increase, especially for geopotential height, but the shape of the growth curve would remain the same (not shown). The vertically average growth rate of Fig. 3 serves for the comparison with the spread growth derived in the modal space later on.

As the stratosphere has usually not been the subject of predictability research, we perform a sensitivity study involving only model levels under 10 hPa. In L91 formulation, there are 73 model levels from 10 hPa and the surface, with the new top level (level 19) located at around 11 hPa. We shall refer to the two sets of results and L91 and L73 spread referring to the whole model atmosphere up to 1 Pa and the atmosphere up to 10 hPa, respectively. This approach is similar to Žagar et al. (2013) who also removed the top levels owing to unrealistic large spread coming from the assimilation of perturbed observations.

### b. Application of NMF to the L91 and L73 ENS dataset

The 3D projection derived in section 2 and applied to 91 and 73 model levels of the ENS system consists of a set of numerical vertical structure functions and horizontal Hough functions, known as Hough harmonics. Solutions of the vertical structure equation in each case apply the top boundary condition *m* = 3 and 4 for L73 case have their lowest zero crossing in the upper troposphere under 200 hPa. Higher modes have a more complex structure in the stratosphere and are characterized by a zig-zag shape moving downward as *m* increases. As discussed in Žagar et al. (2015), a number of vertical modes need to be summed up in order to represent the circulation within a given layer. In this study, the results are shown for all vertical modes summed up as we focus on the horizontal dependence of the spread. The associated equivalent depths vary from about 10 km for

Separation of the spread between the balanced and IG modes is illustrated by an example shown in Fig. 5 that applies to the 1 December 2014 case presented in Figs. 1–3. To present the two components of the spread in physical space, Eqs. (4) and (5) are applied to all 50 members separately to filter the balanced and IG modes in each member and forecast range. Then the spread is computed for the two 50-member ensembles in physical space as in Fig. 1. The results are shown for the initial state, for 24-h and 7-day forecasts. First, Fig. 5 shows that the zonally averaged spread associated with the IG modes in the troposphere and the lower stratosphere equals or exceeds the spread associated with the balanced modes in the initial ensemble. The extratropical spread in the upper stratosphere and above appears mostly balanced. The balanced spread in the midlatitudes grows quickly and in 24-h forecasts the extratropical zonally averaged spread exceeds the tropical spread. In the 7-day range, the tropical IG spread has increased most near the tropopause level; however, it is smaller than the midlatitude IG spread, which has two maxima in the lower and upper troposphere just like the balanced spread.

Figure 6 shows the horizontal distribution of balanced and IG spread at two model levels. One is model level 43, which is on average located at about 134 hPa, and has the maximal tropical spread throughout the forecast. Another presented level is lower down in the troposphere, level 53 at around 256 hPa, where the zonal wind spread is dominated by uncertainties in baroclinic regions. Similar to Fig. 5, Fig. 6 shows that a part of the midlatitude spread associated with baroclinic waves projects onto the IG modes. The explanation is provided by the dynamics of baroclinic waves (e.g., Plougonven and Snyder 2007) and improvements of the ECMWF model capability to represent the IG dynamics. Figures 5 and 6, derived for a single randomly chosen date, are relevant illustrations of the properties of the long-term statistics shown in the next section. In particular, Fig. 6 suggests a prevalence of synoptic and planetary scales of the spread, especially in the tropics. The tropical region within 30° of the equator represents half of the atmosphere analyzed in our globally averaged statistics.

## 4. Modal statistics of the ENS spread

### a. Scale-dependent distribution of spread

The scale-dependent distribution is shown separately for the spread associated with the balanced and IG modes. Global ENS spread shown in Fig. 7 as a function of the zonal wavenumber is obtained by applying Eq. (9) to the outputs of projections for every analyzed time followed by time averaging and summation of the spread over all meridional scales and vertical modes. Spectra are shown every 12 h for forecast lead times 0–168 h. To make the discussion easier, the balanced and IG components are additionally shown together at several time steps. Figure 7 includes spread in the wind components and geopotential on all model levels as well as the spread in surface pressure field. We choose to present the statistics for L91 model but the curves for L73 look similar; the primary difference is in the spread overall scaling (i.e., curves for L73 are, on average, about 40% lower in magnitude but have the same shape).

First, in agreement with Fig. 6, we notice that the globally integrated ensemble spread has the largest magnitudes at the planetary and synoptic scales; this applies to both balanced and IG components. However, there are a number of substantial differences between the balanced and IG spread components. Initially, the IG spread is larger than the balanced spread on all scales but especially at subsynoptic scales (Fig. 7c). As the forecast evolves, the balanced spread in large scales becomes more dominant. After 24 h, the balanced spread dominates on planetary scales while, after 3 days, the balanced spread prevails over the IG spread also in synoptic scales. The crossing scale (wavenumber) at which the balanced ensemble spread becomes smaller than the IG spread is after day 3 at

After the first few days, the increase of spread in subsynoptic scales slows down, especially for the balanced spread. The asymptotic curves are computed from the uncorrelated operational analyses on the same N64, L91 grid and shown by the two top solid black curves in Figs. 7a and 7b. They correspond to the climatological spread as represented by 0000 UTC analyses 9 days apart during winter months in the period 2008–12 for which the L91 model was operational. This period and sampling were chosen to get the size of the sample similar to the size of the ensemble (50 members). In agreement with the spread growth in physical space in Fig. 3, the asymptotic curves in Fig. 7 appear rather distant from the 7-day forecast range, especially in the planetary scale and in scales smaller than zonal wavenumber 70 (a scale of about 200 km in the midlatitudes). A large gap between the ensemble spread and the asymptotic curve in this part of mesoscale range, especially in the IG component, seems difficult to close by the spread growth in longer forecasts. The size of this gap will be quantified later in the paper.

Figure 7 differs from the classical representation of perturbations growing fastest in the smallest scales and propagating to largest scales which are initially well defined. Instead, Fig. 7 shows the integrated information from Figs. 5 and 6; the initial perturbations are largest on the largest scales and globally are the most significant in the tropics. This may be a property of the ECMWF system associated with the method used to generate initial uncertainties. The comparison of initial perturbations with the short-term forecast error variances simulated by the EDA approach (Žagar et al. 2013) suggests that a major portion of initial uncertainties in the tropics comes from EDA; the amplitude of perturbations generated by SVs is smaller in the tropics but the role of SVs is more important in the midlatitudes (Buizza et al. 2008; Magnusson et al. 2008). If Fig. 7c is replaced by the same figure based on L73 data, the gap between the balanced and IG spread at initial time would be even larger, especially on large scales (not shown), in agreement with Fig. 5. The distribution of initial uncertainties and the growth of spread may be different in another global ENS, especially in a system not based on the assimilation of perturbed observations and in a system without a stochastic representation of model error. Magnusson et al. (2008) suggested that differences in results of various ensemble systems (i.e., NCEP, ECMWF, and the Canadian ensemble prediction system) more likely come from differences in their data assimilation and forecast models than from their choice of initial perturbation methods.

It is evident from Fig. 7c that the rate of growth is largest in the balanced modes. This growth rate is quantified in Fig. 8a, which shows the percentage of the balanced spread in every zonal scale divided by the total spread in the same scale and time step. It shows that the balanced and IG spread components are equally represented beyond the zonal wavenumber 15 in forecasts of 4 days and longer. Beyond the 24-h forecast range, the ratio between the balanced and IG spread is nearly constant with value 40:60 in all scales smaller than wavenumber 70. The growth in L73 appears very similar except that the 70% level in planetary scales is reached about a day earlier. The vertical alignment of curves in various scales in Fig. 8a suggests that the portion of balanced spread in these scales would remain constant over a longer integration except in the largest scales which still need to converge toward asymptotic values. The scale-dependent distribution of climatological spread can be estimated from the asymptotic curves shown in Fig. 7. The result is shown in Fig. 8b. The balanced spread makes up nearly all the variance in the zonally averaged state (

The dynamical regimes of the planetary, synoptic, and subsynoptic scales are further examined by splitting the zonal scales into three regimes as in Jung and Leutbecher (2008). Three ranges are defined by the zonal wavenumbers *n*) can be either global or bounded to the tropics depending on the equivalent depth and the zonal wavenumber. On the other hand, meridional modes in the range *n* = 3–15 are primarily associated with the midlatitude circulation. This comes out from the meridional structure of the Hough harmonics as a function of equivalent depth and zonal scale [for details, see Žagar et al. (2015)]. Consequently, the balanced spread grows most in modes *n* = 5–10 and the amplitude is largest in synoptic scales. There is a sharp change between the *n* (modes 0–2) that represent tropical circulation. The largest component of unbalanced spread is the subsynoptic range and its growth is associated with both the midlatitudes and the tropics as illustrated in Fig. 6. The mode

A figure similar to Fig. 9 could be discussed for the vertical distribution of spread, which is left out the present paper because of a more involved discussion of the vertical modes. We only note that in addition to the barotropic mode, the majority of spread is found in the range of vertical modes *m* = 5–15 associated with the midlatitude baroclinic structures and tropical deep convection (figure not shown).

A further insight into the scale-dependent spread growth is obtained if the balanced and IG components of the spread are normalized with their corresponding initial values as presented in Fig. 10. Figure 10 shows that during the first 12 h of forecasts, the spread increases mostly in the smallest scales in both IG and balanced modes. A small increase of spread in synoptic scales in balanced modes can first be noticed in 48-h forecast. To the contrary, synoptic scales of the IG modes, which according to Fig. 10c could be defined by zonal wavenumbers between 4 and 30 (5000–700 km), contain growth already from the initial time. As suggested by Fig. 6, the growth of IG spread is not limited to the tropical synoptic scales but it is a part of synoptic-scale spread also in the midlatitudes. In Fig. 10c this can be diagnosed as a shift of the scale with the maximal spread growth toward larger scales during the forecasts.

If the IG spread is further split between the eastward- and westward-propagating modes, the tropical and midlatitude properties of the spread can be more closely associated with the two directions of propagation of IG modes. First, Fig. 11 shows that the EIG spread grows more than the WIG spread in longer forecasts and in the planetary scales. This feature is associated with the initial uncertainties being in the tropics and tropical dynamics projecting onto the lowest meridional EIG modes, especially the Kelvin mode. The WIG modes are more representative of the IG spread associated with the midlatitudes weather systems; namely, the expansion functions are derived for the state of rest and midlatitudes IG modes move eastward with the baroclinic perturbances. In addition, behavior of spread in WIG modes at small scales is similar to that of balanced modes. In the 7-day forecast range, both balanced and unbalanced growth is largest in wavenumbers 6 and 7. However, the absolute maximum of the growth is in the zonal wavenumber 1 in balanced spread. The planetary scales have almost flat spread growth throughout the forecast; this is only partly associated with the large spread in the stratosphere as the growth curves derived from L73 system look very similar to that shown in Fig. 11 (not shown). The main difference is that the growth in planetary and synoptic balanced modes has a greater magnitude than for the presented L91 case since the growth of spread in these scales is slower in the upper stratosphere and mesosphere than lower down. In Fig. 10 we also find that the balanced spread in small scales appears nearly saturated after 2 days while spread in IG modes steadily grows. A possible reason is attributed to the stochastic model error schemes [stochastically perturbed parameterized tendencies (SPPT) and stochastic kinetic energy backscatter (SKEB)] that may be adding spread primarily in the IG modes. This and other possible reasons should be investigated by separate sensitivity simulations diagnosed in the modal space.

If Fig. 10 is integrated zonally, the resulting 3D global spread growth can be seen in Fig. 12. The left column of Fig. 12 shows the global spread growth normalized by its initial value for different regimes, whereas the right column shows the total spread normalized by its climatological value (top black curves in Fig. 7). The expected difference in the spread growth in various scales based on Figs. 10 and 11 becomes more clear in Fig. 12. First, subsynoptic scales show signs of spread saturation after the first day of forecasts (Fig. 12d). These scales contain the largest percentage of their asymptotic spread in the initial state—around 40%. In the 7-day forecast range, small scales reach 80% climatological spread with balanced spread larger than IG spread (Fig. 12h). In the synoptic and planetary scales the balanced spread growth appears close to linear in the 2–7-day forecast range similar to Kuhl et al. (2007) whereas the total growth in synoptic scales (IG+ROT in Fig. 12c) seems to slow down toward the end of the period (Fig. 12b,c). In terms of climatological limit, the IG spread is 30%–50% of its climatological distribution at the initial time (Fig. 12e–h). Normalized curves also suggest that the subsynoptic IG spread would hardly converge to its asymptotic limit in agreement with Fig. 7. The balanced spread in synoptic and planetary scales appears difficult to extrapolate in Fig. 12 and a longer dataset and a more detailed analysis and curve fitting would be needed. Finally, the globally and scale-averaged growth shown in Fig. 12a can be compared with Fig. 3 to find out that the growth rates are similar. Figures 12d and 12h also show a notable difference between EIG and WIG growth in the subsynoptic scales; here, the WIG growth is greater than the EIG, which is most likely due to the growth rate of the IG spread in midlatitudes exceeding that in the tropics.

If we draw a line at 60% of climatological spread in Figs. 12e and 12h in analogy to the anomaly correlation coefficient used to evaluate potential prediction skill (Murphy and Epstein 1989), we can note several interesting features associated with a different growth in balanced and IG modes. In the subsynoptic scale, the IG spread reaches the 60% spread curve around 36 h into the forecast followed by the balanced spread soon after. In the synoptic scale, IG and balanced spread are crossing the 60% line around the 2.5- and 5-day forecasts, respectively. In the planetary scale, IG spread is at 60% of the climatological value at forecast day 5 while the balanced spread is not close to this limit in 7-day range we have. Finally, when all scales are considered together, the IG spread is at 60% in the 2-day range, the balanced spread in the 5.5-day range, and the total spread in the 4.5-day range. These numbers related IG and balanced dynamics to useful forecast range in the global NWP systems.

### b. Modal view of reliability

Many operational ensembles are characterized by underdispersiveness; that is, their spread is, on average, smaller than the average error of the ensemble mean. In Figs. 13 and 14 we measure ENS reliability in different meridional modes and motion types for the three zonal scale ranges defined earlier. As the control forecast at initial time (used as verification) was available only at 0000 UTC, there are seven analyzed forecast ranges. We will discuss L73 results since the ensemble spread in the top levels above 10 hPa cannot be taken as representative for the comparison with the ensemble-mean error. The difference between the L91 and L73 cases is seen in balanced modes and large scales where it resides in these top levels. The total reliability shown in Fig. 13 suggests that the ECMWF ensemble is somewhat underdispersive in balanced modes in planetary and synoptic scales and in subsynoptic-scale IG modes. The underdispersiveness in the balanced planetary and synoptic scales slowly increases in time. A lack of spread in the IG modes, especially in the EIG modes in subsynoptic scales, seems nearly constant in forecasts. A scale-dependent insight into the missing spread is provided by Fig. 14. It shows that the missing spread in planetary scales is associated with the symmetric balanced modes with small *n*, especially the Kelvin modes (

## 5. Discussion and conclusions

We have presented a new method based on a three-dimensional and multivariate modal analysis to assess ensemble properties, and we have applied it to the ECMWF medium-range ensemble (ENS). In contrast to other approaches, the proposed method considers the global model atmosphere; it integrates the spread in both wind and mass variables across all model levels. It allows a comparison between the spread associated with the balanced and inertio-gravity (IG) modes as well as a quantification of the spread in various horizontal scales and vertical modes. The new approach has been applied to the 1-month (December 2014) dataset of the operational ENS system. Several features of the ensemble system revealed in the modal space are discussed in relation to the initial uncertainties and model properties. Presented results discuss the horizontal and temporal properties of the spread distribution; the interpretation of vertical modes in the ECMWF system with many levels in the stratosphere is more complex and a subject of a separate study.

The following findings are considered the most relevant:

- The initial distribution of spread in ECMWF ENS is dominated by simulated uncertainties in the tropics. Their maximum is in the upper tropical troposphere and under the tropopause, suggesting that they are associated with convection. A large component of the tropical initial perturbations is generated by the EDA method based on the assimilation of perturbed observations. The singular vectors make a smaller portion of the tropical initial perturbations but are crucial in the midlatitudes (Buizza et al. 2008). The global observing system is dominated by the mass-field observations, especially in the tropics; their random perturbations may contribute to noisy analyses since the applied 12-h 4DVAR window and the background-error covariance matrix may not produce balanced analysis solutions in the tropics (Žagar et al. 2004). This suggests that the magnitude of initial perturbations contributed by the EDA method may not be optimal. For the same reason the initial ensemble spread may be projecting more on the IG modes than on the balanced modes on all scales and especially on subsynoptic scales.
- In relation to the large EDA component of initial perturbations and the tropics making a half of the global atmosphere, the scale-dependent distribution of initial uncertainties appears similar to the distribution of short-term forecast errors derived from the ECMWF ensemble data assimilation (Žagar et al. 2013). During the forecasts, the distribution of spread shifts toward the climatological distribution characteristic of the ECMWF analyses. This points out a great dependence of the spread growth on the initial uncertainties.
- Early in the forecasts, the growth of spread takes place in all scales and all modes although the growth is largest in the smallest scales and in the tropical synoptic scales (IG modes). The IG modes have the largest growth in the synoptic scales just like the balanced modes. However, the overall spread growth is dominated by the growth of spread in balanced modes in synoptic and planetary scales in midlatitudes as found in previous studies. The IG spread is closer to its climatological value than the balanced spread in each wavenumber and forecast range except in the subsynoptic scales. The subsynoptic range is characterized by a gap between the IG and balanced spread and their asymptotic curves that seems hard to close during longer forecasts. Globally integrated IG and balanced spread reach 60% of their asymptotic values after 2 and 5 days of forecasts, respectively.
- The component of spread projecting onto IG modes is found both in the tropics and in the midlatitudes where the IG spread accompanies the balanced spread associated with the baroclinic systems. In midlatitudes, the IG spread is more associated with the westward-propagating IG modes while the tropical spread projects more strongly to the eastward-propagating IG modes. Presented modal analysis method provides a tool for the quantification of the spread associated with the IG wave sources in the model in midlatitudes at any forecast time in relation to the quantification of the variability due to IG dynamics. Similarly, it allows the quantification of the role of various equatorial waves in the simulated tropical variability.
- Reliability is defined in modal space by verifying ensemble forecasts against control analyses to provide reliability along the three spatial dimensions and for the two motion types. Applied to the ECMWF system, it suggests that the underdispersiveness of the system is not large and it is associated primarily with the lack of tropical variability, especially a lack of variability due to Kelvin waves across all scales.

Very similar results were obtained for another season and longer datasets from the previous ENS system using L62. This does not mean that presented results apply to other ensemble systems. It is suggested that the 3D analysis based on NMF representation is applied routinely to ensemble forecasts to diagnose their performance in a more complete way and identify which areas need more attention. For example, this work suggests that in the ECMWF system more attention should be given to how initial perturbations are simulated in the tropical area and to the effect that the model error schemes have in exciting divergence among the ensemble members during the forecast evolution.

## Acknowledgments

This work has been carried out under the funding from the European Research Council (ERC), Grant Agreement 280153. The applied NMF software MODES is available at http://meteo.fmf.uni-lj.si/MODES. Paul Dando of ECMWF is thanked for his crucial support in the extraction of ENS model-level data in real time. We are grateful to Akira Kasahara and an anonymous reviewer for their comments on the manuscript. In particular, we thank Akira Kasahara for the discussion about the computation of the ensemble variance. Blaž Jesenko is thanked for improving Figs. 1, 5, and 6.

## APPENDIX

### The ECMWF Medium-Range/Monthly Ensemble (ENS)

*j*goes from 0 to 50, with 0 identifying the control forecast, run from the unperturbed analysis and without model error schemes. The ENS runs twice per day, at 0000 and 1200 UTC, up to 15 days, and twice per week up to 32 days (at 0000 UTC every Monday and Thursday).

_{L}1279L137 resolution and with a 12-h assimilation window, interpolated from the T

_{L}1279L137 resolution to the T

_{L}639L91 ensemble resolution. The perturbations are defined by a linear combination of singular vectors (SVs; Buizza and Palmer 1995) and perturbations defined by the ECMWF ensemble of data assimilations (EDA; Buizza et al. 2008; Isaksen et al. 2010):where

The SVs, the fastest-growing perturbations over a 48-h time interval, are computed at T42L91 resolution. SVs optimized to have maximum total-energy growth over the different areas, are linearly combined and scaled to have amplitudes, locally, comparable to analysis error estimates provided by the ECMWF high-resolution 4DVAR.

The EDA-based perturbation is defined by the difference between a 6-h forecast started from an EDA member completed 6 h earlier (*d* − 6) and the mean of all the EDA forecasts. The reason why differences between forecasts are used is because the EDA runs in delay mode, so only 6-h forecasts from the latest one are available at the time the ensemble starts production.

Each EDA member is generated by an independent 4DVAR with a lower resolution (T_{L}399) than the high-resolution 4DVAR (T_{L}1279). Each EDA member uses perturbed observations, with the observations’ perturbations sampled from a Gaussian distribution with zero mean and the observation error standard deviation. Each EDA member nonlinear trajectory is generated using also the stochastically perturbed parameterized tendencies (SPPT) scheme to simulate model uncertainties (see below for a description of the SPPT scheme). Since November 2013, the EDA has been including 25 independent 4DVAR run at T_{L}399L137 resolution and with a 12-h assimilation window (the EDA only had 11 members before November 2013).

The EDA-based perturbations are defined by differences between 6-h forecasts started from the most recent available EDA analyses (these analyses are valid for 6 h earlier than the ENS initial time). Differences are computed between each of the 25 perturbed forecasts and their ensemble mean, and the 25 perturbations are added to and subtracted from the unperturbed analysis. SV- and EDA-based perturbations are defined such that full symmetry is maintained in the ENS initial perturbations [i.e. even member

The ocean initial conditions are defined by the five-member ensemble of ocean analyses, produced by the NEMO three-dimensional variational assimilation system (NEMOVAR; Mogensen et al. 2012b). Each ocean analysis is generated using all available in situ temperature and salinity data, an estimate of the surface forcing from ECMWF short-range atmospheric forecasts, sea surface temperature analysis, and satellite altimetry measurements. One member is generated using unperturbed wind forcing provided by the high-resolution 4DVAR, while the other four members are generated using perturbed versions of the unperturbed wind forcing. Model uncertainties are simulated only in the free atmosphere (i.e., not in the land surface, nor in the ocean), using two stochastic schemes: the stochastically perturbed parameterized tendency (SPPT; Buizza et al. 1999; Palmer et al. 2009) and the backscatter (SKEB; Shutts 2005; Berner et al. 2009) schemes. SPPT is designed to simulate random model errors due to parameterized physical processes; the current version uses three spatial- and time-level perturbations. SKEB is designed to simulate the upscale energy transfer induced by the unresolved scales on the resolved scales.

Since March 2008, when the ECMWF medium-range and monthly ensembles were joined, a key component of the ECMWF ensemble used to generate some bias-corrected and/or calibrated products has been the reforecast suite (Vitart et al. 2008; Leutbecher and Palmer 2008). This suite includes a five-member ensemble run once per week with the operational configuration (resolution, model cycle, etc.) for the past 20 years. These reforecasts are used to estimate the model climate required to generate some ensemble products (e.g., the extreme forecast index, or weekly average anomaly maps) and to calibrate the ENS forecasts.

The two most recent changes introduced in ENS were as follows:

- November 2013—The top of the model was raised from 5 to 0.1 hPa, the number of vertical levels was increased from 62 to 91, and the atmosphere and ocean models were coupled from initial time (rather than from day 10 as was done up to that time). Furthermore, surface initial perturbations were added to the ENS initial perturbations [see section 4.3 in Balsamo et al. (2014)], and the coupling to the ocean model NEMO was moved forward in time, from forecast day 10 to the initial time. These changes, together with major upgrades in the model parameterizations, led to further improvements of the ensemble skill, especially in the longer forecast range.
- May 2015—The monthly integrations have been extended from 32 to 46 days, and the number of members of the reforecast suite has been increased from 5 to 22: 11 members are now generated for the past 20 years twice a week, in correspondence with the Monday and Thursday monthly extensions.

## REFERENCES

Balsamo, G., and et al. , 2014: Representing the Earth surfaces in the Integrated Forecasting System: Recent advances and future challenges. ECMWF Tech. Memo. 729, 50 pp. [Available online at http://old.ecmwf.int/publications/library/ecpublications/_pdf/tm/701-800/tm729.pdf.]

Berner, J., , G. J. Shutts, , M. Leutbecher, , and T. Palmer, 2009: A spectral stochastic kinetic energy backscatter scheme and its impact on flow-dependent predictability in the ECMWF ensemble prediction system.

,*J. Atmos. Sci.***66**, 603–626, doi:10.1175/2008JAS2677.1.Brier, M., 1950: Verification of forecasts expressed in terms of probability.

,*Mon. Wea. Rev.***78**, 1–3, doi:10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2.Buizza, R., 2010: Horizontal resolution impact on short- and long-range forecast error.

*Quart. J. Roy. Meteor. Soc.*,**136**, 1020–1035, doi:10.1002/qj.613.Buizza, R., 2014: The TIGGE global, medium-range ensembles. ECMWF Tech. Memo. 739, 53 pp. [Available online at http://old.ecmwf.int/publications/library/ecpublications/_pdf/tm/701-800/tm739.pdf.]

Buizza, R., , and T. Palmer, 1995: The singular-vector structure of the atmospheric global circulation.

,*J. Atmos. Sci.***52**, 1434–1456, doi:10.1175/1520-0469(1995)052<1434:TSVSOT>2.0.CO;2.Buizza, R., , M. Miller, , and T. N. Palmer, 1999: Stochastic representation of model uncertainties in the ECMWF EPS.

,*Quart. J. Roy. Meteor. Soc.***125**, 2887–2908, doi:10.1002/qj.49712556006.Buizza, R., , P. L. Houtekamer, , G. Pellerin, , Z. Toth, , Y. Zhu, , and M. Wei, 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems.

,*Mon. Wea. Rev.***133**, 1076–1097, doi:10.1175/MWR2905.1.Buizza, R., , M. Leutbecher, , and L. Isaksen, 2008: Potential use of an ensemble of analyses in the ECMWF Ensemble Prediction System.

,*Quart. J. Roy. Meteor. Soc.***134**, 2051–2066, doi:10.1002/qj.346.Candille, G., , and O. Talagrand, 2005: Evaluation of probabilistic prediction systems for a scalar variable.

,*Quart. J. Roy. Meteor. Soc.***131**, 2131–2150, doi:10.1256/qj.04.71.Hagedorn, R., , R. Buizza, , T. Hamill, , M. Leutbecher, , and T. N. Palmer, 2012: Comparing TIGGE multimodel forecasts with reforecast calibrated ECMWF ensemble forecasts.

,*Quart. J. Roy. Meteor. Soc.***138**, 1814–1827, doi:10.1002/qj.1895.Hough, S. S., 1898: On the application of harmonic analysis to the dynamical theory of the tides—Part II. On the general integration of Laplace’s dynamical equations.

,*Philos. Trans. Roy. Soc. London***191A**, 139–185, doi:10.1098/rsta.1898.0005.Isaksen, L., , M. Bonavita, , R. Buizza, , M. Fisher, , J. Haseler, , M. Leutbecher, , and L. Raynaud, 2010: Ensemble of data assimilations at ECMWF. ECMWF Tech. Memo. 636, 48 pp. [Available online at http://old.ecmwf.int/publications/library/ecpublications/_pdf/tm/601-700/tm636.pdf.]

Janssen, P., , J.-R. Bidlot, , S. Abdalla, , and H. Hersbach, 2005: Progress in ocean wave forecasting at ECMWF. ECMWF Tech. Memo. 478, 27 pp. [Available online http://old.ecmwf.int/publications/library/ecpublications/_pdf/tm/401-500/tm478.pdf.]

Janssen, P., and et al. , 2013: Air-sea interaction and surface waves. ECMWF Tech. Memo. 712, 34 pp. [Available online http://old.ecmwf.int/publications/library/ecpublications/_pdf/tm/701-800/tm712.pdf.]

Jung, T., , and M. Leutbecher, 2008: Scale-dependent verification of ensemble forecasts.

,*Quart. J. Roy. Meteor. Soc.***134**, 973–984, doi:10.1002/qj.255.Kasahara, A., 1976: Normal modes of ultralong waves in the atmosphere.

,*Mon. Wea. Rev.***104**, 669–690, doi:10.1175/1520-0493(1976)104<0669:NMOUWI>2.0.CO;2.Kasahara, A., 1978: Further studies on a spectral model of the global barotropic primitive equations with Hough harmonic expansions.

,*J. Atmos. Sci.***35**, 2043–2051, doi:10.1175/1520-0469(1978)035<2043:FSOASM>2.0.CO;2.Kasahara, A., , and K. Puri, 1981: Spectral representation of three-dimensional global data by expansion in normal mode functions.

,*Mon. Wea. Rev.***109**, 37–51, doi:10.1175/1520-0493(1981)109<0037:SROTDG>2.0.CO;2.Kuhl, D., and et al. , 2007: Assessing predictability with a local ensemble Kalman filter.

,*J. Atmos. Sci.***64**, 1116–1140, doi:10.1175/JAS3885.1.Leutbecher, M., , and T. N. Palmer, 2008: Ensemble forecasting.

,*J. Comput. Phys.***227**, 3515–3539, doi:10.1016/j.jcp.2007.02.014.Lorenz, E. N., 1982: Atmospheric predictability experiments with a large numerical model.

,*Tellus***34**, 505–513, doi:10.1111/j.2153-3490.1982.tb01839.x.Magnusson, L., , M. Leutbecher, , and E. Kallen, 2008: Comparison between singular vectors and breeding vectors as initial perturbations for the ECMWF ensemble prediction system.

,*Mon. Wea. Rev.***136**, 4092–4104, doi:10.1175/2008MWR2498.1.Mogensen, K., , M. A. Balmaseda, , and A. Weaver, 2012a: The NEMOVAR ocean data assimilation system as implemented in the ECMWF ocean analysis for System 4. ECMWF Tech. Memo. 668, 59 pp. [Available online http://old.ecmwf.int/publications/library/ecpublications/_pdf/tm/601-700/tm668.pdf.]

Mogensen, K., , S. Keeley, , and P. Towers, 2012b: Coupling of the NEMO and IFS models in a single executable. ECMWF Tech. Memo. 673, 23 pp. [Available online at http://old.ecmwf.int/publications/library/ecpublications/_pdf/tm/601-700/tm673.pdf.]

Murphy, A. H., , and E. S. Epstein, 1989: Skill scores and correlation coefficients in model verification.

,*Mon. Wea. Rev.***117**, 572–582, doi:10.1175/1520-0493(1989)117<0572:SSACCI>2.0.CO;2.Palmer, T. N., , R. Buizza, , F. Doblas-Reyes, , T. Jung, , M. Leutbeche, , G. J. Shutts, , M. Steinheimer, , and A. Weisheimer, 2009: Stochastic parametrization and model uncertainty. ECMWF Tech. Memo. 598, 42 pp. [Available online at http://old.ecmwf.int/publications/library/ecpublications/_pdf/tm/501-600/tm598.pdf.]

Phillips, N. A., 1957: A coordinate system having some special advantages for numerical forecasting.

,*J. Meteor.***14**, 184–185, doi:10.1175/1520-0469(1957)014<0184:ACSHSS>2.0.CO;2.Phillips, N. A., 1990: Dispersion processes in large-scale weather prediction. Sixth IMO Lecture, WMO 700, 126 pp.

Plougonven, R., , and C. Snyder, 2007: Inertia–gravity waves spontaneously generated by jets and fronts. Part I: Different baroclinic life cycles.

,*J. Atmos. Sci.***64**, 2502–2520, doi:10.1175/JAS3953.1.Saetra, O., , J.-R. Bidlot, , H. Hersbach, , and D. Richardson, 2002: Effects of observations errors on the statistics for ensemble spread and reliability. ECMWF Tech. Memo. 393, 12 pp. [Available online http://old.ecmwf.int/publications/library/ecpublications/_pdf/tm/301-400/tm393.pdf.]

Shutts, G. J., 2005: A kinetic energy backscatter algorithm for use in ensemble prediction systems.

,*Quart. J. Roy. Meteor. Soc.***131**, 3079–3100, doi:10.1256/qj.04.106.Tribbia, J., , and D. P. Baumhefner, 2004: Scale interactions and atmospheric predictability: An updated perspective.

,*Mon. Wea. Rev.***132**, 703–713, doi:10.1175/1520-0493(2004)132<0703:SIAAPA>2.0.CO;2.Vitart, F., and et al. , 2008: The new VarEPS-monthly forecasting system: A first step towards seamless prediction.

,*Quart. J. Roy. Meteor. Soc.***134**, 1789–1799, doi:10.1002/qj.322.Wilks, D., 2011:

*Statistical Methods in the Atmospheric Sciences.*3rd ed. Elsevier, 676 pp.Žagar, N., , N. Gustafsson, , and E. Källén, 2004: Variational data assimilation in the tropics: The impact of a background error constraint.

,*Quart. J. Roy. Meteor. Soc.***130**, 103–125, doi:10.1256/qj.03.13.Žagar, N., , J. Tribbia, , J. L. Anderson, , and K. Raeder, 2009: Uncertainties of estimates of inertio–gravity energy in the atmosphere. Part I: Intercomparison of four analysis datasets.

*Mon. Wea. Rev.*,**137**, 3837–3857, doi:10.1175/2009MWR2815.1; Corrigendum,**138**, 2476–2477, doi:10.1175/2010MWR3256.1.Žagar, N., , L. Isaksen, , D. Tan, , and J. Tribbia, 2013: Balance and flow-dependency of background-error variances in the ECMWF 4D-Var ensemble.

*Quart. J. Roy. Meteor. Soc.*,**139**, 1229–1238, doi:10.1002/qj.2033.Žagar, N., , A. Kasahara, , K. Terasaki, , J. Tribbia, , and H. Tanaka, 2015: Normal-mode function representation of global 3D datasets: Open-access software for the atmospheric research community.

,*Geosci. Model Dev.***8**, 1169–1195, doi:10.5194/gmd-8-1169-2015.