1. Introduction
Data assimilation produces the initial conditions of numerical weather forecasts by a statistical interpolation of the atmospheric observations (e.g., Szunyogh 2014). Thus, the initial conditions have an inherently random error component, which we will call the analysis uncertainty. The amplification of the analysis uncertainty by the chaotic dynamics of the atmosphere would lead to an inevitable growth of the magnitude of the forecast uncertainty, even if models were perfect replica of the atmosphere. Because models are not perfect and use statistical considerations to account for processes at the unresolved scales, model errors and model uncertainty also contribute to the forecast uncertainty.
In perhaps the most influential paper ever written on the dynamics of forecast error (uncertainty) growth, Lorenz (1969b) investigated the role of scale interactions in the error growth process. He argued that forecast errors saturated, predictability was lost, at increasingly larger scales as forecast time increased. Lorenz’s results were most recently revisited by Tribbia and Baumhefner (2004), Rotunno and Snyder (2008), and Durran and Gingrich (2014). Tribbia and Baumhefner (2004) augmented Lorenz’s description of the process by adding that in the extratropics the dominant errors asymptoted to the baroclinically active scales, where they then grew exponentially. Rotunno and Snyder (2008) replaced the original two-dimensional vorticity equation in Lorenz’s model by the surface geostrophic equation. They pointed out that the rapid downscale propagation of errors at the mesoscales played an important role in the rapid saturation of the forecast errors at the smaller scales. Durran and Gingrich (2014) extended this argument to emphasize that synoptic-scale errors, even if they had small magnitude, led to a rapid saturation of the errors at the smaller scales.
In their ensemble-based predictability studies, Kuhl et al. (2007) and Satterfield and Szunyogh (2010, 2011) (hereafter referred to collectively as KSS) observed the same spectral evolution of the forecast uncertainty as Tribbia and Baumhefner (2004). They also found that a 40–80-member ensemble of forecasts was efficient in capturing the dominant synoptic-scale patterns of forecast uncertainty. The present study extends the investigations of KSS to the THORPEX Interactive Grand Global Ensemble (TIGGE), which comprises operational global ensemble forecast data from the major operational centers. The primary objective of the paper is to verify that the findings of KSS also hold for the operational ensemble forecast systems. The results shown later strongly suggest the affirmative. The secondary objective of this paper is to provide information about the performance of the operational ensemble forecast systems.
The outline of the paper is the following. Section 2 provides background information about the TIGGE dataset, while section 3 describes the local diagnostics we employ. Section 4 describes the dynamics of the atmosphere for the investigated time period, while section 5 presents the results of our diagnostic calculations. Section 6 shows further diagnostic results for a select group of ensembles, and section 7 offers our conclusions.
2. The TIGGE dataset
We provide a general description of the TIGGE dataset and briefly discuss the techniques that the centers use to represent the effects of initial condition and model uncertainties.
a. The dataset
The TIGGE dataset is a collection of global ensemble forecasts from the major NWP centers (Bougeault et al. 2010; Swinbank et al. 2016). The goal of TIGGE is to provide ensemble data to support both academic research and operational product development. The forecasts are collected in real time and made available to the scientific community by the archiving centers in an easily accessible uniform format.
We analyze data from the following forecast centers: the European Centre for Medium-Range Weather Forecasts (ECMWF), the National Centers for Environmental Prediction (NCEP), the Met Office (UKMO), the China Meteorological Administration (CMA), the Japan Meteorological Agency (JMA), the Korean Meteorological Administration (KMA), the Meteorological Service of Canada (CMC), and Météo-France.
We do not consider data from two of the NWP centers, the Australian Bureau of Meteorology (BoM) and the Centro de Previsão de Tempo e Estudos Climáticos (CPTEC), which provide data to the TIGGE dataset: data were not available from the BoM ensemble for the time period of this study, while CPTEC discovered an error in their ensemble and was planning to regenerate the data.
b. Initial condition perturbations
The degrees of freedom of the dynamics of an operational model is orders of magnitude larger than the operationally attainable number of ensemble members. The forecast centers have addressed this challenging aspect of ensemble forecasting by developing techniques for the generation of initial condition perturbations that efficiently represent the growing part of the analysis (initial condition) errors. Table 1 shows a list of the ensemble generation techniques of the different centers.
Ensemble forecast systems included from TIGGE.
The bred vector method (Toth and Kalnay 1993, 1997), which was originally developed and implemented at NCEP, is currently used by CMA and KMA. To create the bred vectors, the analysis is randomly perturbed and the full nonlinear model is run for a short period (e.g., 6 h) for both the control (unperturbed) and perturbed analyses. The control forecast is subtracted from the perturbed forecasts and the resulting perturbations are rescaled to the magnitude of the initial perturbations. The cycle is repeated by adding the rescaled perturbations to the next analysis. After several days of “breeding,” growing patterns dominate the spatiotemporal evolution of the perturbations.
Another type of initial condition perturbations, which is used by ECMWF, JMA, and Météo-France are known as (right) singular vectors (Buizza et al. 1993; Molteni and Palmer 1993; Mureau et al. 1993); these vectors are the initial perturbations that grow fastest with respect to a preselected norm and optimization (forecast) time. For the ensembles included in this study, the norm is a quadratic norm with energy dimension (e.g., Buizza et al. 1993) and the optimization time is 48-h forecast hours. The Météo-France ensemble uses a combination of singular vectors and evolved singular vectors, where the evolved singular vectors are created such that the analysis time, for which the initial perturbations are created, coincides with the end of the optimization period. The evolved singular vectors are hoped to represent analysis uncertainties that were likely to grow in the analysis cycles of the immediate past. ECMWF also used evolved singular vectors in the past, but by the time of the present study they have switched to using an ensemble of data assimilations (EDA) to account for the error growth during the previous data assimilation cycles (Buizza et al. 2008). To create these perturbations, observations are perturbed randomly in accordance with their presumed error statistics in the data assimilation system; each set of perturbed observations is assimilated into a different ensemble member.
The method currently used for the generation of ensemble perturbations at NCEP is similar to the generation of bred vectors, but it uses information from the data assimilation system to determine a spatiotemporally varying rescaling factor. This method is called ensemble transform with rescaling (ETR) and was developed by Wei et al. (2008); ensemble perturbations valid for the analysis time are obtained through an ensemble transform of a previous set of forecast perturbations, taking into account the observation error statistics and centering the perturbations on the analysis.
The Met Office uses a local ensemble transform Kalman filter (ETKF) to generate their perturbations (Bishop et al. 2001; Wang and Bishop 2003; Bowler and Mylne 2009). The largest difference between the ETKF and the ETR methods is that the ETKF produces an ensemble of full analyses rather than an ensemble of rescaled perturbations. The analysis perturbations, which are obtained by taking the difference between the members of the analysis ensemble and the mean of the analysis ensemble, are added to the operational 4D-Var analysis to obtain the ensemble of perturbed initial conditions. Last, CMC uses an ensemble Kalman filter (EnKF) to generate the analysis ensemble. Unlike at UKMO, their ensemble is centered on the mean analysis produced by the EnKF.
c. Model error parameterization techniques
In addition to chaotic model dynamics acting on uncertain initial conditions, model errors also contribute to the forecast error growth. Model errors affect the forecasts continuously during the entire forecast period. They also contribute to the initial conditions uncertainty through the forecast phase of the analysis cycles.
The main sources of model errors are thought to be the parameterization schemes for the subgrid processes. One technique to account for these sources is the method of stochastically perturbed parameterization tendencies (SPPT; Buizza et al. 1999; Palmer et al. 2009). This technique perturbs the total contribution of the parameterized processes to the tendency of the state variables in the model. Another technique is to use different parameterization schemes for the same processes, or to use different values of the prescribed parameters of the parameterization schemes. This approach is known as the multiphysics technique (Berner et al. 2011; Houtekamer 2002).
The effects of uncertainties injected at the smallest resolved scales cannot be directly simulated by the models, because the interactions between those scales and the larger scales are distorted by the models: some scale interactions are explicitly eliminated by the truncation strategies, while others are eliminated by dampening the smaller-scale motions. Time integration schemes also contribute to the diffusiveness of the models at scales where nature is not diffusive. An approach to make the representation of the effect of upscale propagating uncertainties by the ensemble more realistic, called stochastic energy backscattering (SKEB), was introduced and described by Shutts (2005, 2013), Berner et al. (2009), Bowler et al. (2009), Charron et al. (2010), and Tennant et al. (2011). One other method used to account for this uncertainty is called stochastic time tendency perturbations (STTP; Hou et al. 2008). Whereas SKEB focuses on subgrid-scale error, STTP adds stochastic forcing at all scales. Since SPPT/multiphysics and SKEB/STTP simulate different aspects of model error dynamics, they can be used in conjunction. This practice is followed at both ECMWF and CMC.
3. Local diagnostics
An ensemble forecast provides a flow (synoptic situation) dependent prediction of the probability distribution of the forecast uncertainty. We focus on examining the mean and the covariance matrix of the predicted probability distribution. The significance of the covariance matrix of the distribution is that it describes both the structure and the magnitude of the predicted uncertain flow features. In addition, under the assumption that the probability distribution of the uncertainty is Gaussian, the mean and the covariance matrix together provide a complete description of the predicted probability distribution.
a. Local vectors and their covariance
b. Diagnostics for the predicted magnitude of the uncertainty
1) Optimality conditions
The vector 
2) The magnitude of the forecast uncertainty
3) The uncertainty in the proxy for the true state
The presence of 
4) The error in the prediction of the mean
5) Lorenz curves
In our application of Eq. (16), F is either 
We emphasize that Eq. (16) is a crude parameterization of the function that describes the error growth process. It is based on the assumption that the initial error growth process is linear and nonlinear effects become important only later, once the magnitude of the errors becomes sufficiently large. While this is a reasonable assumption for the error growth at the synoptic scales, it is clearly violated by the rapidly saturating errors at the smaller scale. In addition, earlier studies (e.g., Orrell et al. 2001; Vannitsem and Toth 2002; Nicolis et al. 2009) also demonstrated that model errors tended to lead to a nonlinear short-term error growth. Hence, Eq. (16) is expected to provide a better description of the error growth process from the forecast times at which synoptic-scale errors become dominant; and the estimates of β should be considered a particularly crude estimate.
c. Diagnostics for the predicted structure of the uncertainties
Finally, it should be noted that there are approaches different than ours to define an orthogonal basis for the investigation of the evolution of ensemble perturbations. For instance, Leutbecher and Lang (2014) defined an orthogonal basis by the leading right singular vectors of the tangent linear version of the model, while Zagar et al. (2015) defined an orthogonal basis by the normal mode functions consisting of vertical structure functions, each associated with a set of horizontal Hough functions.
d. Estimation of the expected value
We estimate the expected value by either an average over all forecasts of the same lead time and all locations 
We compute diagnostics for forecasts that were started between 0000 UTC 1 January and 1800 UTC 29 February 2012. The diagnostics are computed for the entire forecast range of each ensemble system. Diagnostics that require the estimation of temporal means are computed by taking averages over all forecasts of equal forecast time. Spatiotemporal means for the NH extratropics are computed by averaging the temporal means over all locations between 30° and 75°N.
4. Experiment design
The local volume 
We use ECMWF analyses as the proxy 
5. The atmospheric flow
For the time period of our investigation, we describe the synoptic-scale transients with the help of the eddy kinetic energy equation (EKE) and the low-frequency transients by the zonal anomalies of the seasonal mean flow.
High-frequency (synoptic scale) transient components of the flow
The first term of the right-hand side describes the horizontal eddy kinetic energy transport, the second term is the geopotential flux convergence, the third term is the baroclinic energy conversion, and the fourth term is the barotropic energy conversion. Term five describes the vertical eddy kinetic energy transport through the bottom and the top surfaces, while term six represents the transport of eddy potential energy through the same surfaces. Finally, the last term is the residue term that represents the bulk effect of the errors of the numerical calculations and all processes unaccounted for by the other terms. The most important such process is dissipation, which usually makes the residue term negative. Because not all variables necessary for the computation of the terms of the eddy kinetic energy equation are available in the TIGGE dataset, we use data from the ERA-Interim reanalysis for the description of the flow. Unlike the previous diagnostics that are calculated on local volumes, the eddy kinetic energy equation is calculated on the global grid.
The computation of the terms of the eddy kinetic energy equation starts with a decomposition of the spatiotemporally evolving atmospheric state variables into a spatially varying time-mean component and a spatiotemporally evolving eddy component. We compute the time mean for January–March, because even though all forecasts start in January and February, some of them end in March. The time-mean component of the geopotential height field at the 500-Pa pressure level is shown in Fig. 1 for both the ECMWF and NCEP analyses: the time-mean flow has a dominantly zonal wavenumber-2 structure, with negative zonal anomalies in the Pacific and the Atlantic storm-track regions and positive zonal anomalies in the exit regions of the storm tracks.
Zonal anomalies of the time-mean flow. Color shades indicate the zonal anomalies for the investigated time period based on the (left) ECMWF analyses and (right) NCEP analyses, while contours show the time-mean flow (geopotential height) at the 500-hPa level based on the same analyses.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Zonal anomalies of the time-mean flow. Color shades indicate the zonal anomalies for the investigated time period based on the (left) ECMWF analyses and (right) NCEP analyses, while contours show the time-mean flow (geopotential height) at the 500-hPa level based on the same analyses.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Zonal anomalies of the time-mean flow. Color shades indicate the zonal anomalies for the investigated time period based on the (left) ECMWF analyses and (right) NCEP analyses, while contours show the time-mean flow (geopotential height) at the 500-hPa level based on the same analyses.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Figure 2 shows the time-mean eddy kinetic energy (top-left panel) and the time mean of the three terms of the eddy kinetic energy that dominate the changes in the time-mean eddy kinetic energy (other three panels). These terms represent baroclinic energy conversion (top-right panel), barotropic energy conversion (bottom-left panel), and the horizontal transport of the eddy kinetic energy (bottom-right panel). The largest local maxima of the eddy kinetic energy are located in the eastern sector of the Pacific storm track. These maxima are due to the local generation of kinetic energy by baroclinic energy conversion and the transport of eddy kinetic energy generated upstream by baroclinic energy conversion.
The time mean of the eddy kinetic energy conversion processes for the investigated time period. Color shades show the time mean of the (top left) eddy kinetic energy (J), (top right) baroclinic energy conversion (J day−1), (bottom left) barotropic energy conversion (J day−1), and (bottom right) horizontal transport of the eddy kinetic energy (J day−1).
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
The time mean of the eddy kinetic energy conversion processes for the investigated time period. Color shades show the time mean of the (top left) eddy kinetic energy (J), (top right) baroclinic energy conversion (J day−1), (bottom left) barotropic energy conversion (J day−1), and (bottom right) horizontal transport of the eddy kinetic energy (J day−1).
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
The time mean of the eddy kinetic energy conversion processes for the investigated time period. Color shades show the time mean of the (top left) eddy kinetic energy (J), (top right) baroclinic energy conversion (J day−1), (bottom left) barotropic energy conversion (J day−1), and (bottom right) horizontal transport of the eddy kinetic energy (J day−1).
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Barotropic energy conversion is the (nonlinear) transfer of kinetic energy between the synoptic-scale transients and the seasonal mean flow. Where it is positive, kinetic energy is transferred to the synoptic-scale eddies, while where it is negative, kinetic energy is transferred to the seasonal mean flow. While kinetic energy is transferred to the seasonal mean flow in the exit regions of the storm tracks, kinetic energy is transferred to the synoptic-scale eddies over North America and western Europe.
A comparison of Figs. 2 and 1 show the close relationship between the high- and low-frequency transients: baroclinic energy conversion at the synoptic scales is the most intense in regions of the negative anomalies of the time-mean flow, while (negative) barotropic energy conversion from the synoptic to the large scales plays a direct role in the slow changes of the large-scale flow in regions of positive anomalies. These nonlinear barotropic energy conversion processes control the energy transfer between the high-frequency transients and the slowly varying large-scale flow.
6. Results on the predictions of the magnitude of the uncertainty
In this section, our attention is focused on studying the relationship between the evolutions of 
a. The evolution of 
1) Diagnostics based on averages over all forecasts and locations
We first examine the evolutions of 
The evolution of the diagnostics—
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
The evolution of the diagnostics—
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
The evolution of the diagnostics—
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
The evolution of the diagnostics—VS, TV, TVS and 
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
The evolution of the diagnostics—VS, TV, TVS and
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
The evolution of the diagnostics—VS, TV, TVS and
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
For most ensembles, 
The ensemble for which the gap between 
While there are more pronounced differences in the evolution of 
2) Sensitivity of the results to the choice of the proxy for the true state
To test the robustness of the diagnostic results shown in Figs. 3 and 4 to the choice of the proxy 
Illustration of the robustness of the results of Figs. 3 and 4 to the choice of the proxy for the true state. The diagnostics are shown for the UKMO ensemble for the cases in which the proxy for the true state is defined by (left) the ECMWF analyses and (right) the NCEP analyses.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Illustration of the robustness of the results of Figs. 3 and 4 to the choice of the proxy for the true state. The diagnostics are shown for the UKMO ensemble for the cases in which the proxy for the true state is defined by (left) the ECMWF analyses and (right) the NCEP analyses.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Illustration of the robustness of the results of Figs. 3 and 4 to the choice of the proxy for the true state. The diagnostics are shown for the UKMO ensemble for the cases in which the proxy for the true state is defined by (left) the ECMWF analyses and (right) the NCEP analyses.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
3) Diagnostics based on averages over all forecasts
To save space, we show temporally averaged forms of the diagnostics only for selected ensembles. Figures 6 and 7 show 
Spatial distribution of the average forecast uncertainty for the ECMWF ensemble. Shown are 
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spatial distribution of the average forecast uncertainty for the ECMWF ensemble. Shown are
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spatial distribution of the average forecast uncertainty for the ECMWF ensemble. Shown are
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
As in Fig. 6, but for the CMC ensemble.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
As in Fig. 6, but for the CMC ensemble.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
As in Fig. 6, but for the CMC ensemble.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Figures 8 and 9 show 
Spatial distribution of the average ensemble spread for the ECMWF ensemble. Shown are 
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spatial distribution of the average ensemble spread for the ECMWF ensemble. Shown are
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spatial distribution of the average ensemble spread for the ECMWF ensemble. Shown are
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
As in Fig. 8, but for the CMC ensemble.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
As in Fig. 8, but for the CMC ensemble.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
As in Fig. 8, but for the CMC ensemble.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spatial distribution of the mean uncertainty for the ECMWF ensemble. Shown are M (color shades) and the time mean of the geopotential analyses at 500 hPa (contours) at analysis and three different forecast times. Dashes indicate the southern boundary of the region used for the computation of the spatial averages of Fig. 3 and 4.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spatial distribution of the mean uncertainty for the ECMWF ensemble. Shown are M (color shades) and the time mean of the geopotential analyses at 500 hPa (contours) at analysis and three different forecast times. Dashes indicate the southern boundary of the region used for the computation of the spatial averages of Fig. 3 and 4.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spatial distribution of the mean uncertainty for the ECMWF ensemble. Shown are M (color shades) and the time mean of the geopotential analyses at 500 hPa (contours) at analysis and three different forecast times. Dashes indicate the southern boundary of the region used for the computation of the spatial averages of Fig. 3 and 4.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spaghetti diagram for the ensemble of mean forecasts. The mean forecasts were obtained by averaging each member of the ECMWF ensemble over the investigated time period. Shown by gray contour lines are the 5350-gpm isohypses for the ensemble members. The black contour line shows the time mean of the ECMWF analyses for the investigated time period.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spaghetti diagram for the ensemble of mean forecasts. The mean forecasts were obtained by averaging each member of the ECMWF ensemble over the investigated time period. Shown by gray contour lines are the 5350-gpm isohypses for the ensemble members. The black contour line shows the time mean of the ECMWF analyses for the investigated time period.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spaghetti diagram for the ensemble of mean forecasts. The mean forecasts were obtained by averaging each member of the ECMWF ensemble over the investigated time period. Shown by gray contour lines are the 5350-gpm isohypses for the ensemble members. The black contour line shows the time mean of the ECMWF analyses for the investigated time period.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spaghetti diagrams for select ensemble systems at the 360-h lead time. The mean forecasts were obtained by averaging each member of the ensemble over the investigated time period. Shown by gray contour lines are the 5350-gpm isohypses for the ensemble members. The black contour line shows the time mean of the ECMWF analyses for the investigated time period.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spaghetti diagrams for select ensemble systems at the 360-h lead time. The mean forecasts were obtained by averaging each member of the ensemble over the investigated time period. Shown by gray contour lines are the 5350-gpm isohypses for the ensemble members. The black contour line shows the time mean of the ECMWF analyses for the investigated time period.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spaghetti diagrams for select ensemble systems at the 360-h lead time. The mean forecasts were obtained by averaging each member of the ensemble over the investigated time period. Shown by gray contour lines are the 5350-gpm isohypses for the ensemble members. The black contour line shows the time mean of the ECMWF analyses for the investigated time period.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
b. Spectral evolution of 
We illustrate the spectral evolution of the forecast uncertainty and the ensemble spread with the example of the ECMWF and the CMC ensembles. Figure 13 shows the spectral evolution of the two quantities for the meridional component of the wind vector at 500 hPa. The left panels show the evolution of the spectral distribution of 
Spectral evolution of the forecast uncertainty and the ensemble spread for the meridional component of the wind vector at 500 hPa for the ECMWF and CMC ensembles. Shown are the meridional averages of the zonal power spectra of the meridional wind associated with (left) the ensemble spread and (right) the forecast uncertainty. The lowest curves show the spectra at analysis time, while the other curves show the spectra with 2-day increments of the forecast time (the top curves are for day-14 forecast time). The red curve shows the linear regression of the maximum power and the associated zonal wavenumber for all of the forecast times.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spectral evolution of the forecast uncertainty and the ensemble spread for the meridional component of the wind vector at 500 hPa for the ECMWF and CMC ensembles. Shown are the meridional averages of the zonal power spectra of the meridional wind associated with (left) the ensemble spread and (right) the forecast uncertainty. The lowest curves show the spectra at analysis time, while the other curves show the spectra with 2-day increments of the forecast time (the top curves are for day-14 forecast time). The red curve shows the linear regression of the maximum power and the associated zonal wavenumber for all of the forecast times.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spectral evolution of the forecast uncertainty and the ensemble spread for the meridional component of the wind vector at 500 hPa for the ECMWF and CMC ensembles. Shown are the meridional averages of the zonal power spectra of the meridional wind associated with (left) the ensemble spread and (right) the forecast uncertainty. The lowest curves show the spectra at analysis time, while the other curves show the spectra with 2-day increments of the forecast time (the top curves are for day-14 forecast time). The red curve shows the linear regression of the maximum power and the associated zonal wavenumber for all of the forecast times.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
The evolution of the spectra of 
The difference in the spectra of the ensemble spread between the two ensembles (left panels) is the largest at analysis time: the shape of the spectra of 
c. Qualitative description of the forecast uncertainty growth process
The information provided by Figs. 13, 6, 7, and 10 suggests the following general description of the forecast uncertainty growth process in the NH extratropics:
the initial growth is the fastest at the synoptic scales that are most sensitive to baroclinic instability;
the growing, and later saturating, synoptic-scale features of uncertainty fill the region that extends from the entrance region of the Pacific storm track to the exit region of the Atlantic storm track;
because the position and the spatial structure of the storm-track regions are controlled by the slowly varying large-scale flow, the low-frequency transients have a major influence on the spatiotemporal distribution of the forecast uncertainty;
as uncertainties start saturating at the scales most sensitive to baroclinic instability, the wavenumber of dominant instability gradually shifts toward the larger scales (lower wavenumbers); and
at the subsynoptic scales (zonal wavenumbers larger than about 12–14), the uncertainty saturates as predicted by Lorenz’s theory.
The diagnostic results for the ensemble spread (Figs. 8, 9, 10, and 13) suggest that all ensemble forecast systems can capture the main characteristics of the error growth process. Figures 3 and 4 show, however, that the differences between the models, analysis systems and ensemble generation techniques have important effects on the accuracy of the quantitative prediction of the uncertainty.
d. Lorenz curve–based analysis of the evolution of 
1) Estimation of the parameters
2) Comparison of the estimated parameters of the Lorenz curves
The estimated parameters of the Lorenz curves for the forecast uncertainty (
Estimates of the parameters of the Lorenz curves for the different ensembles. The crisscross (×) indicates parameters for which the estimation process failed, while italics indicate estimates that most likely have unusually large errors.
Lorenz curves for the forecast uncertainty (
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Lorenz curves for the forecast uncertainty (
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Lorenz curves for the forecast uncertainty (
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Lorenz curves for the ensemble spread (
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Lorenz curves for the ensemble spread (
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Lorenz curves for the ensemble spread (
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
To make the interpretation of the values of α in Table 2 more transparent, the table also includes the values of 
We recall from section 3b(5) that we consider the parameter β to be a measure of the contribution of model errors to the growth of the forecast uncertainty 
The most interesting conclusions that can be drawn about β are the following. An inefficient representation of the model error forcing by the ensemble is the most likely main source of the underestimation of the forecast uncertainty by the NCEP ensemble (top-right panel of Fig. 3). The good balance between 
For comparison, we also computed the parameters of Lorenz curves for the more conventional choice of the 500-hPa geopotential height rather than a combination of the virtual temperature and the two horizontal wind components in the layer between the 1000- and 200-hPa levels. In these calculations, we computed 
Estimates of the parameters of the Lorenz curves for the different ensembles using 500-hPa geopotential height without localization in the calculation of 
3) The relationship between 
We also prepared Lorenz curve–style figures for 
Estimates of 
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Estimates of
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Estimates of
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
7. Results on predictions of the structure of uncertainty
We examine the evolution of 
a. Diagnostics based on averages over all forecasts and locations
For the examination of the spatiotemporally averaged form of 
The generally small differences between 
For most ensembles, the asymptotic value of 
A small difference between 
A comparison of the two left panels of Fig. 3 suggests that the outstanding performance of the ECMWF ensemble is due to a combination of a faster convergence of 
b. Diagnostics based on averages over all forecasts
Figures 17 and 18 show the spatiotemporal evolution of 
Spatiotemporal evolution of 
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spatiotemporal evolution of
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
Spatiotemporal evolution of
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
As in Fig. 17, but for the CMC ensemble.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
As in Fig. 17, but for the CMC ensemble.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
As in Fig. 17, but for the CMC ensemble.
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
A particularly good example for the ensembles capturing a large part of the structure of the most important local uncertainty is the matching pair of local maxima in 
An overinflation of 
The spatiotemporal evolution of the local ratio between 
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
The spatiotemporal evolution of the local ratio between
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
The spatiotemporal evolution of the local ratio between
Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1
8. Conclusions
We proposed a description of the forecast uncertainty growth process summarized by an itemized list in section 6c, which emphasizes the earlier rapid growth of uncertainties at the synoptic scales and the later shift of the dominant error growth toward the large scales in the spatiotemporal evolution of forecast uncertainty. We found that the TIGGE ensembles were able to capture the main characteristics of the error growth process. The results also showed, however, that the accuracy of the quantitative prediction of the forecast uncertainty was strongly system dependent. While the best performing ensembles did well with respect to the spatiotemporally averaged diagnostics, the location-dependent temporally averaged diagnostics revealed that even the best performing ensembles had large errors in the representation of the local properties of the uncertainty.
A result that was particularly interesting from both a theoretical and a practical point of view was the typical growth of the error in the prediction of the mean state with increasing forecast time. Such a drift of the predicted probability distribution of the state in state space greatly reduces the utility of the longer range (week 2 and beyond) ensemble predictions. If the growth of the mean forecast uncertainty is primarily due to shortcomings of the models and/or the ensemble generation techniques, there is hope that it can be greatly reduced by refining the models and the ensemble generation techniques. However, if it is due to some fundamental properties of the atmospheric dynamics, it may turn out to be a major barrier to the extension of numerical weather forecasts into the subseasonal-to-seasonal forecast range.
Our analysis showed that the growing errors in the prediction of the mean state were dominantly due to errors in the prediction of low-frequency changes in the large-scale flow. One school of thoughts suggests that low-frequency variability is a manifestation of the internal (chaotic) variability of the atmospheric dynamics at the large scales (e.g., Legras and Ghil 1985). If this was true and the models fully captured the internal variability of the atmosphere, the systematic errors in the prediction of the mean state would not increase with forecast time and the ensembles were able to capture the related forecast uncertainty. Hence, one potential explanation for our result is that the models cannot fully capture the internal variability of the atmosphere; for instance, due to poor representation of the atmospheric dynamics in the tropics and/or atmosphere–ocean interactions. The results of Reynolds et al. (2015), for example, suggest that ensemble forecasts tend to lose temporal variability with increasing forecast time. Another potential explanation is that the low-frequency variability is not dominated by internal variability of the atmospheric dynamics. In particular, some authors (e.g., Sura et al. 2005) have argued that low- frequency variability may be due to state-dependent variations of stochastic feedbacks. Such state-dependent stochastic feedback in a long-range prediction is provided by the high-frequency (synoptic scale) transients.
Acknowledgments
This study was supported by the National Science Foundation (Grant ATM-AGS-1237613), while the TIGGE data were downloaded from the ECMWF data portal.
REFERENCES
Berner, J., G. J. Shutts, M. Leutbecher, and T. N. Palmer, 2009: A spectral stochastic kinetic energy backscatter scheme and its impact on flow-dependent predictability in the ECMWF ensemble prediction system. J. Atmos. Sci., 66, 603–626, doi:10.1175/2008JAS2677.1.
Berner, J., S. Y. Ha, J. P. Hacker, A. Fournier, and C. Snyder, 2011: Model uncertainty in a mesoscale ensemble prediction system: Stochastic versus multiphysics representations. Mon. Wea. Rev., 139, 1972–1995, doi:10.1175/2010MWR3595.1.
Bishop, C., B. Etherton, and S. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129, 420–436, doi:10.1175/1520-0493(2001)129<0420:ASWTET>2.0.CO;2.
Bougeault, P., and Coauthors, 2010: The THORPEX Interactive Grand Global Ensemble (TIGGE). Bull. Amer. Meteor. Soc., 91, 1059–1072, doi:10.1175/2010BAMS2853.1.
Bowler, N. E., and K. Mylne, 2009: Ensemble transform Kalman filter perturbations for a regional ensemble prediction system. Quart. J. Roy. Meteor. Soc., 135, 757–766, doi:10.1002/qj.404.
Bowler, N. E., A. Arribas, S. E. Beare, K. R. Mylne, and G. J. Shutts, 2009: The local ETKF and SKEB: Upgrades to the MOGREPS short-range ensemble prediction system. Quart. J. Roy. Meteor. Soc., 135, 767–776, doi:10.1002/qj.394.
Buizza, R., J. Tribbia, F. Molteni, and T. Palmer, 1993: Computation of optimal unstable structures for a numerical weather prediction model. Tellus, 45A, 388–407, doi:10.1034/j.1600-0870.1993.t01-4-00005.x.
Buizza, R., M. Milleer, and T. N. Palmer, 1999: Stochastic representation of model uncertainties in the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 125, 2887–2908, doi:10.1002/qj.49712556006.
Buizza, R., M. Leutbecher, and L. Isaksen, 2008: Potential use of an ensemble of analyses in the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 134, 2051–2066, doi:10.1002/qj.346.
Chang, E., 2000: Wave packets and life cycles of troughs in the upper troposphere. Part II: Seasonal and hemispheric variations. Mon. Wea. Rev., 128, 25–50, doi:10.1175/1520-0493(2000)128<0025:WPALCO>2.0.CO;2.
Charron, M., G. Pellerin, L. Spacek, P. L. Houtekamer, N. Gagnon, H. L. Mitchell, and L. Michelin, 2010: Toward random sampling of model error in the Canadian ensemble prediction system. Mon. Wea. Rev., 138, 1877–1901, doi:10.1175/2009MWR3187.1.
Dalcher, A., and E. Kalnay, 1987: Error growth and predictability in operational ECMWF forecasts. Tellus, 39A, 474–491, doi:10.1111/j.1600-0870.1987.tb00322.x.
Durran, D., and M. Gingrich, 2014: Atmospheric predictability: Why butterflies are not of practical importance. J. Atmos. Sci., 71, 2476–2488, doi:10.1175/JAS-D-14-0007.1.
Epstein, E., 1969: Stochastic dynamic prediction. Tellus, 21A, 739–759, doi:10.1111/j.2153-3490.1969.tb00483.x.
Hou, D., Z. Toth, Y. Zhu, and W. Yang, 2008: Impact of a stochastic perturbation scheme on NCEP global ensemble forecast system. 19th Conf. on Probability and Statistics, New Orleans, LA, Amer. Meteor. Soc., 1.1. [Available online at https://ams.confex.com/ams/pdfpapers/134165.pd.]
Houtekamer, P. L., 2002: The use of multiple parameterizations in ensembles. Proc. ECMWF Workshop on Model Uncertainty, Reading, United Kingdom, ECMWF, 163–174.
Hunt, B., E. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230, 112–126, doi:10.1016/j.physd.2006.11.008.
Kuhl, D., and Coauthors, 2007: Assessing predictability with a local ensemble Kalman filter. J. Atmos. Sci., 64, 1116–1140, doi:10.1175/JAS3885.1.
Legras, B., and M. Ghil, 1985: Persistent anomalies, blocking and variations in atmospheric predictability. J. Atmos. Sci., 42, 433–471, doi:10.1175/1520-0469(1985)042<0433:PABAVI>2.0.CO;2.
Leith, C., 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev., 102, 409–418, doi:10.1175/1520-0493(1974)102<0409:TSOMCF>2.0.CO;2.
Leutbecher, M., and S. T. K. Lang, 2014: On the reliability of ensemble variance in subspaces defined by singular vectors. Quart. J. Roy. Meteor. Soc., 140, 1453–1466, doi:10.1002/qj.2229.
Lorenz, E. N., 1969a: Atmospheric predictability as revealed by naturally occurring analogues. J. Atmos. Sci., 26, 636–646, doi:10.1175/1520-0469(1969)26<636:APARBN>2.0.CO;2.
Lorenz, E. N., 1969b: The predictability of a flow which possesses many scales of motion. Tellus, 21A, 289–307, doi:10.1111/j.2153-3490.1969.tb00444.x.
Lorenz, E. N., 1982: Atmospheric predictability experiments with a large numerical model. Tellus, 34A, 505–513, doi:10.1111/j.2153-3490.1982.tb01839.x.
Magnusson, L., and E. Kallen, 2013: Factors influencing the skill improvements in the ECMWF forecasting system. Mon. Wea. Rev., 141, 3142–3153, doi:10.1175/MWR-D-12-00318.1.
Molteni, F., and T. Palmer, 1993: Predictability and finite-time instability of the northern winter circulation. Quart. J. Roy. Meteor. Soc., 119, 269–298, doi:10.1002/qj.49711951004.
Mureau, R., F. Molteni, and T. Palmer, 1993: Ensemble prediction using dynamically conditioned perturbations. Quart. J. Roy. Meteor. Soc., 119, 299–323, doi:10.1002/qj.49711951005.
Nicolis, C., P. Perdigao, and S. Vannitsem, 2009: Dynamics of prediction errors under the combined effect of initial condition and model errors. J. Atmos. Sci., 66, 766–778, doi:10.1175/2008JAS2781.1.
Orlanski, I., and J. Katzfey, 1991: The life cycle of a cyclone wave in the Southern Hemisphere. Part I: Eddy energy budget. J. Atmos. Sci., 48, 1972–1998, doi:10.1175/1520-0469(1991)048<1972:TLCOAC>2.0.CO;2.
Orlanski, I., and E. Chang, 1993: Ageostrophic geopotential fluxes and upstream development of baroclinic waves. J. Atmos. Sci., 50, 212–225, doi:10.1175/1520-0469(1993)050<0212:AGFIDA>2.0.CO;2.
Orrell, D., L. Smith, J. Barkmeijer, and T. Palmer, 2001: Model error in weather forecasting. Nonlinear Processes Geophys., 8, 357–371, doi:10.5194/npg-8-357-2001.
Ott, E., and Coauthors, 2004: A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A, 415–428, doi:10.1111/j.1600-0870.2004.00076.x.
Palmer, T. N., R. Buizza, F. Doblas-Reyes, T. Jung, M. Leutbecher, G. J. Shutts, M. Steinheimer, and A. Weisheimer, 2009: Stochastic parameterization and model uncertainty. ECMWF Tech. Memo. 598, European Centre for Medium-Range Weather Forecasts, 42 pp.
Reynolds, C., E. A. Satterfield, and C. H. Bishop, 2015: Using forecast temporal variability to evaluate model behavior. Mon. Wea. Rev., 143, 4785–4804, doi:10.1175/MWR-D-15-0083.1.
Rotunno, R., and C. Snyder, 2008: A generalization of Lorenz’s model for the predictability of flows with many scales of motion. J. Atmos. Sci., 65, 1063–1076, doi:10.1175/2007JAS2449.1.
Satterfield, E., and I. Szunyogh, 2010: Predictability of the performance of an ensemble forecast system: Predictability of the space of uncertainties. Mon. Wea. Rev., 138, 962–981, doi:10.1175/2009MWR3049.1.
Satterfield, E., and I. Szunyogh, 2011: Assessing the performance of an ensemble forecast system in predicting the magnitude and the spectrum of analysis and forecast uncertainties. Mon. Wea. Rev., 139, 1207–1223, doi:10.1175/2010MWR3439.1.
Shutts, G. J., 2005: A kinetic energy backscatter algorithm for use in ensemble prediction systems. Quart. J. Roy. Meteor. Soc., 131, 3079–3102, doi:10.1256/qj.04.106.
Shutts, G. J., 2013: Coarse graining the vorticity equation in the ECMWF integrated forecasting system: The search for kinetic energy backscatter. J. Atmos. Sci., 70, 1233–1241, doi:10.1175/JAS-D-12-0216.1.
Sura, P., M. Newman, C. Penland, and P. Sardeshmukh, 2005: Multiplicative noise and non-Gaussianity: A paradigm for atmospheric regimes? J. Atmos. Sci., 62, 1391–1409, doi:10.1175/JAS3408.1.
Swinbank, R., and Coauthors, 2016: The TIGGE project and its achievements. Bull. Amer. Meteor. Soc., 97, 49–67, doi:10.1175/BAMS-D-13-00191.1.
Szunyogh, I., 2014: Applicable Atmospheric Dynamics: Techniques for the Exploration of Atmospheric Dynamics. 1st ed. World Scientific, 608 pp.
Szunyogh, I., E. Kostelich, G. Gyarmati, D. Patil, B. Hunt, E. Kalnay, E. Ott, and J. Yorke, 2005: Assessing a local ensemble Kalman filter: Perfect model experiments with the National Centers for Environmental Prediction global model. Tellus, 57A, 528–545, doi:10.1111/j.1600-0870.2005.00136.x.
Szunyogh, I., E. Kostelich, G. Gyarmati, E. Kalnay, B. Hunt, E. Ott, E. Satterfield, and J. Yorke, 2008: A local ensemble transform Kalman filter data assimilation system for the NCEP global model. Tellus, 60A, 113–130, doi:10.1111/j.1600-0870.2007.00274.x.
Talagrand, O., 1981: A study of the dynamics of four-dimensional data assimilation. Tellus, 33A, 43–60, doi:10.1111/j.2153-3490.1981.tb01729.x.
Tennant, W. J., G. J. Shutts, A. Arribas, and S. A. Thompson, 2011: Using a stochastic kinetic energy backscatter scheme to improve MOGREPS probabilistic forecast skill. Mon. Wea. Rev., 139, 1190–1206, doi:10.1175/2010MWR3430.1.
Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74, 2317–2330, doi:10.1175/1520-0477(1993)074<2317:EFANTG>2.0.CO;2.
Toth, Z., and E. Kalnay, 1997: Ensemble forecasting at NCEP and the breeding method. Mon. Wea. Rev., 125, 3297–3319, doi:10.1175/1520-0493(1997)125<3297:EFANAT>2.0.CO;2.
Tribbia, J. J., and D. P. Baumhefner, 2004: Scale interactions and atmospheric predictability: An updated perspective. Mon. Wea. Rev., 132, 703–713, doi:10.1175/1520-0493(2004)132<0703:SIAAPA>2.0.CO;2.
Vannitsem, S., and Z. Toth, 2002: Short-term dynamics of model errors. J. Atmos. Sci., 59, 2594–2604, doi:10.1175/1520-0469(2002)059<2594:STDOME>2.0.CO;2.
Wang, X., and C. Bishop, 2003: A comparison of breeding and ensemble transform Kalman filter ensemble forecast schemes. J. Atmos. Sci., 60, 1140–1158, doi:10.1175/1520-0469(2003)060<1140:ACOBAE>2.0.CO;2.
Wei, M., Z. Toth, R. Wobus, and Y. Zhu, 2008: Initial perturbations based on the ensemble transform (ET) technique in the NCEP global operational forecast system. Tellus, 60A, 62–79, doi:10.1111/j.1600-0870.2007.00273.x.
Zagar, N., R. Buizza, and J. Tribbia, 2015: A three-dimensional multivariate modal analysis of atmospheric predictability with application to the ECMWF ensemble. J. Atmos. Sci., 72, 4423–4444, doi:10.1175/JAS-D-15-0061.1.
For instance, the underestimation of 
