• Berner, J., G. J. Shutts, M. Leutbecher, and T. N. Palmer, 2009: A spectral stochastic kinetic energy backscatter scheme and its impact on flow-dependent predictability in the ECMWF ensemble prediction system. J. Atmos. Sci., 66, 603626, doi:10.1175/2008JAS2677.1.

    • Search Google Scholar
    • Export Citation
  • Berner, J., S. Y. Ha, J. P. Hacker, A. Fournier, and C. Snyder, 2011: Model uncertainty in a mesoscale ensemble prediction system: Stochastic versus multiphysics representations. Mon. Wea. Rev., 139, 19721995, doi:10.1175/2010MWR3595.1.

    • Search Google Scholar
    • Export Citation
  • Bishop, C., B. Etherton, and S. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129, 420436, doi:10.1175/1520-0493(2001)129<0420:ASWTET>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Bougeault, P., and Coauthors, 2010: The THORPEX Interactive Grand Global Ensemble (TIGGE). Bull. Amer. Meteor. Soc., 91, 10591072, doi:10.1175/2010BAMS2853.1.

    • Search Google Scholar
    • Export Citation
  • Bowler, N. E., and K. Mylne, 2009: Ensemble transform Kalman filter perturbations for a regional ensemble prediction system. Quart. J. Roy. Meteor. Soc., 135, 757766, doi:10.1002/qj.404.

    • Search Google Scholar
    • Export Citation
  • Bowler, N. E., A. Arribas, S. E. Beare, K. R. Mylne, and G. J. Shutts, 2009: The local ETKF and SKEB: Upgrades to the MOGREPS short-range ensemble prediction system. Quart. J. Roy. Meteor. Soc., 135, 767776, doi:10.1002/qj.394.

    • Search Google Scholar
    • Export Citation
  • Buizza, R., J. Tribbia, F. Molteni, and T. Palmer, 1993: Computation of optimal unstable structures for a numerical weather prediction model. Tellus, 45A, 388407, doi:10.1034/j.1600-0870.1993.t01-4-00005.x.

    • Search Google Scholar
    • Export Citation
  • Buizza, R., M. Milleer, and T. N. Palmer, 1999: Stochastic representation of model uncertainties in the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 125, 28872908, doi:10.1002/qj.49712556006.

    • Search Google Scholar
    • Export Citation
  • Buizza, R., M. Leutbecher, and L. Isaksen, 2008: Potential use of an ensemble of analyses in the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 134, 20512066, doi:10.1002/qj.346.

    • Search Google Scholar
    • Export Citation
  • Chang, E., 2000: Wave packets and life cycles of troughs in the upper troposphere. Part II: Seasonal and hemispheric variations. Mon. Wea. Rev., 128, 2550, doi:10.1175/1520-0493(2000)128<0025:WPALCO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Charron, M., G. Pellerin, L. Spacek, P. L. Houtekamer, N. Gagnon, H. L. Mitchell, and L. Michelin, 2010: Toward random sampling of model error in the Canadian ensemble prediction system. Mon. Wea. Rev., 138, 18771901, doi:10.1175/2009MWR3187.1.

    • Search Google Scholar
    • Export Citation
  • Dalcher, A., and E. Kalnay, 1987: Error growth and predictability in operational ECMWF forecasts. Tellus, 39A, 474491, doi:10.1111/j.1600-0870.1987.tb00322.x.

    • Search Google Scholar
    • Export Citation
  • Durran, D., and M. Gingrich, 2014: Atmospheric predictability: Why butterflies are not of practical importance. J. Atmos. Sci., 71, 24762488, doi:10.1175/JAS-D-14-0007.1.

    • Search Google Scholar
    • Export Citation
  • Epstein, E., 1969: Stochastic dynamic prediction. Tellus, 21A, 739759, doi:10.1111/j.2153-3490.1969.tb00483.x.

  • Hou, D., Z. Toth, Y. Zhu, and W. Yang, 2008: Impact of a stochastic perturbation scheme on NCEP global ensemble forecast system. 19th Conf. on Probability and Statistics, New Orleans, LA, Amer. Meteor. Soc., 1.1. [Available online at https://ams.confex.com/ams/pdfpapers/134165.pd.]

  • Houtekamer, P. L., 2002: The use of multiple parameterizations in ensembles. Proc. ECMWF Workshop on Model Uncertainty, Reading, United Kingdom, ECMWF, 163174.

  • Hunt, B., E. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230, 112126, doi:10.1016/j.physd.2006.11.008.

    • Search Google Scholar
    • Export Citation
  • Kuhl, D., and Coauthors, 2007: Assessing predictability with a local ensemble Kalman filter. J. Atmos. Sci., 64, 11161140, doi:10.1175/JAS3885.1.

    • Search Google Scholar
    • Export Citation
  • Legras, B., and M. Ghil, 1985: Persistent anomalies, blocking and variations in atmospheric predictability. J. Atmos. Sci., 42, 433471, doi:10.1175/1520-0469(1985)042<0433:PABAVI>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Leith, C., 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev., 102, 409418, doi:10.1175/1520-0493(1974)102<0409:TSOMCF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Leutbecher, M., and S. T. K. Lang, 2014: On the reliability of ensemble variance in subspaces defined by singular vectors. Quart. J. Roy. Meteor. Soc., 140, 14531466, doi:10.1002/qj.2229.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1969a: Atmospheric predictability as revealed by naturally occurring analogues. J. Atmos. Sci., 26, 636646, doi:10.1175/1520-0469(1969)26<636:APARBN>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1969b: The predictability of a flow which possesses many scales of motion. Tellus, 21A, 289307, doi:10.1111/j.2153-3490.1969.tb00444.x.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1982: Atmospheric predictability experiments with a large numerical model. Tellus, 34A, 505513, doi:10.1111/j.2153-3490.1982.tb01839.x.

    • Search Google Scholar
    • Export Citation
  • Magnusson, L., and E. Kallen, 2013: Factors influencing the skill improvements in the ECMWF forecasting system. Mon. Wea. Rev., 141, 31423153, doi:10.1175/MWR-D-12-00318.1.

    • Search Google Scholar
    • Export Citation
  • Molteni, F., and T. Palmer, 1993: Predictability and finite-time instability of the northern winter circulation. Quart. J. Roy. Meteor. Soc., 119, 269298, doi:10.1002/qj.49711951004.

    • Search Google Scholar
    • Export Citation
  • Mureau, R., F. Molteni, and T. Palmer, 1993: Ensemble prediction using dynamically conditioned perturbations. Quart. J. Roy. Meteor. Soc., 119, 299323, doi:10.1002/qj.49711951005.

    • Search Google Scholar
    • Export Citation
  • Nicolis, C., P. Perdigao, and S. Vannitsem, 2009: Dynamics of prediction errors under the combined effect of initial condition and model errors. J. Atmos. Sci., 66, 766778, doi:10.1175/2008JAS2781.1.

    • Search Google Scholar
    • Export Citation
  • Orlanski, I., and J. Katzfey, 1991: The life cycle of a cyclone wave in the Southern Hemisphere. Part I: Eddy energy budget. J. Atmos. Sci., 48, 19721998, doi:10.1175/1520-0469(1991)048<1972:TLCOAC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Orlanski, I., and E. Chang, 1993: Ageostrophic geopotential fluxes and upstream development of baroclinic waves. J. Atmos. Sci., 50, 212225, doi:10.1175/1520-0469(1993)050<0212:AGFIDA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Orrell, D., L. Smith, J. Barkmeijer, and T. Palmer, 2001: Model error in weather forecasting. Nonlinear Processes Geophys., 8, 357371, doi:10.5194/npg-8-357-2001.

    • Search Google Scholar
    • Export Citation
  • Ott, E., and Coauthors, 2004: A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A, 415428, doi:10.1111/j.1600-0870.2004.00076.x.

    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., R. Buizza, F. Doblas-Reyes, T. Jung, M. Leutbecher, G. J. Shutts, M. Steinheimer, and A. Weisheimer, 2009: Stochastic parameterization and model uncertainty. ECMWF Tech. Memo. 598, European Centre for Medium-Range Weather Forecasts, 42 pp.

  • Reynolds, C., E. A. Satterfield, and C. H. Bishop, 2015: Using forecast temporal variability to evaluate model behavior. Mon. Wea. Rev., 143, 47854804, doi:10.1175/MWR-D-15-0083.1.

    • Search Google Scholar
    • Export Citation
  • Rotunno, R., and C. Snyder, 2008: A generalization of Lorenz’s model for the predictability of flows with many scales of motion. J. Atmos. Sci., 65, 10631076, doi:10.1175/2007JAS2449.1.

    • Search Google Scholar
    • Export Citation
  • Satterfield, E., and I. Szunyogh, 2010: Predictability of the performance of an ensemble forecast system: Predictability of the space of uncertainties. Mon. Wea. Rev., 138, 962981, doi:10.1175/2009MWR3049.1.

    • Search Google Scholar
    • Export Citation
  • Satterfield, E., and I. Szunyogh, 2011: Assessing the performance of an ensemble forecast system in predicting the magnitude and the spectrum of analysis and forecast uncertainties. Mon. Wea. Rev., 139, 12071223, doi:10.1175/2010MWR3439.1.

    • Search Google Scholar
    • Export Citation
  • Shutts, G. J., 2005: A kinetic energy backscatter algorithm for use in ensemble prediction systems. Quart. J. Roy. Meteor. Soc., 131, 30793102, doi:10.1256/qj.04.106.

    • Search Google Scholar
    • Export Citation
  • Shutts, G. J., 2013: Coarse graining the vorticity equation in the ECMWF integrated forecasting system: The search for kinetic energy backscatter. J. Atmos. Sci., 70, 12331241, doi:10.1175/JAS-D-12-0216.1.

    • Search Google Scholar
    • Export Citation
  • Sura, P., M. Newman, C. Penland, and P. Sardeshmukh, 2005: Multiplicative noise and non-Gaussianity: A paradigm for atmospheric regimes? J. Atmos. Sci., 62, 13911409, doi:10.1175/JAS3408.1.

    • Search Google Scholar
    • Export Citation
  • Swinbank, R., and Coauthors, 2016: The TIGGE project and its achievements. Bull. Amer. Meteor. Soc., 97, 4967, doi:10.1175/BAMS-D-13-00191.1.

    • Search Google Scholar
    • Export Citation
  • Szunyogh, I., 2014: Applicable Atmospheric Dynamics: Techniques for the Exploration of Atmospheric Dynamics. 1st ed. World Scientific, 608 pp.

  • Szunyogh, I., E. Kostelich, G. Gyarmati, D. Patil, B. Hunt, E. Kalnay, E. Ott, and J. Yorke, 2005: Assessing a local ensemble Kalman filter: Perfect model experiments with the National Centers for Environmental Prediction global model. Tellus, 57A, 528545, doi:10.1111/j.1600-0870.2005.00136.x.

    • Search Google Scholar
    • Export Citation
  • Szunyogh, I., E. Kostelich, G. Gyarmati, E. Kalnay, B. Hunt, E. Ott, E. Satterfield, and J. Yorke, 2008: A local ensemble transform Kalman filter data assimilation system for the NCEP global model. Tellus, 60A, 113130, doi:10.1111/j.1600-0870.2007.00274.x.

    • Search Google Scholar
    • Export Citation
  • Talagrand, O., 1981: A study of the dynamics of four-dimensional data assimilation. Tellus, 33A, 4360, doi:10.1111/j.2153-3490.1981.tb01729.x.

    • Search Google Scholar
    • Export Citation
  • Tennant, W. J., G. J. Shutts, A. Arribas, and S. A. Thompson, 2011: Using a stochastic kinetic energy backscatter scheme to improve MOGREPS probabilistic forecast skill. Mon. Wea. Rev., 139, 11901206, doi:10.1175/2010MWR3430.1.

    • Search Google Scholar
    • Export Citation
  • Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74, 23172330, doi:10.1175/1520-0477(1993)074<2317:EFANTG>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Toth, Z., and E. Kalnay, 1997: Ensemble forecasting at NCEP and the breeding method. Mon. Wea. Rev., 125, 32973319, doi:10.1175/1520-0493(1997)125<3297:EFANAT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Tribbia, J. J., and D. P. Baumhefner, 2004: Scale interactions and atmospheric predictability: An updated perspective. Mon. Wea. Rev., 132, 703713, doi:10.1175/1520-0493(2004)132<0703:SIAAPA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Vannitsem, S., and Z. Toth, 2002: Short-term dynamics of model errors. J. Atmos. Sci., 59, 25942604, doi:10.1175/1520-0469(2002)059<2594:STDOME>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Wang, X., and C. Bishop, 2003: A comparison of breeding and ensemble transform Kalman filter ensemble forecast schemes. J. Atmos. Sci., 60, 11401158, doi:10.1175/1520-0469(2003)060<1140:ACOBAE>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Wei, M., Z. Toth, R. Wobus, and Y. Zhu, 2008: Initial perturbations based on the ensemble transform (ET) technique in the NCEP global operational forecast system. Tellus, 60A, 6279, doi:10.1111/j.1600-0870.2007.00273.x.

    • Search Google Scholar
    • Export Citation
  • Zagar, N., R. Buizza, and J. Tribbia, 2015: A three-dimensional multivariate modal analysis of atmospheric predictability with application to the ECMWF ensemble. J. Atmos. Sci., 72, 44234444, doi:10.1175/JAS-D-15-0061.1.

    • Search Google Scholar
    • Export Citation
  • View in gallery
    Fig. 1.

    Zonal anomalies of the time-mean flow. Color shades indicate the zonal anomalies for the investigated time period based on the (left) ECMWF analyses and (right) NCEP analyses, while contours show the time-mean flow (geopotential height) at the 500-hPa level based on the same analyses.

  • View in gallery
    Fig. 2.

    The time mean of the eddy kinetic energy conversion processes for the investigated time period. Color shades show the time mean of the (top left) eddy kinetic energy (J), (top right) baroclinic energy conversion (J day−1), (bottom left) barotropic energy conversion (J day−1), and (bottom right) horizontal transport of the eddy kinetic energy (J day−1).

  • View in gallery
    Fig. 3.

    The evolution of the diagnostics—, , , and —in the forecasts for four of the ensembles averaged over the NH extratropics and all forecasts started between 1 Jan and 29 Feb 2012. It should be noted that the models have different max forecast lead times.

  • View in gallery
    Fig. 4.

    The evolution of the diagnostics—VS, TV, TVS and —in the forecasts for the remaining four ensembles averaged over the Northern Hemisphere extratropics and all forecasts started between 1 Jan and 29 Feb 2012. It should be noted that the models have different max forecast lead times.

  • View in gallery
    Fig. 5.

    Illustration of the robustness of the results of Figs. 3 and 4 to the choice of the proxy for the true state. The diagnostics are shown for the UKMO ensemble for the cases in which the proxy for the true state is defined by (left) the ECMWF analyses and (right) the NCEP analyses.

  • View in gallery
    Fig. 6.

    Spatial distribution of the average forecast uncertainty for the ECMWF ensemble. Shown are (color shades) and the time mean of the geopotential analyses at 500 hPa (contours) at analysis and three different forecast times. Dashes indicate the southern boundary of the region used for the computation of the spatial averages of Figs. 3 and 4.

  • View in gallery
    Fig. 7.

    As in Fig. 6, but for the CMC ensemble.

  • View in gallery
    Fig. 8.

    Spatial distribution of the average ensemble spread for the ECMWF ensemble. Shown are (color shades) and the time mean of the geopotential analyses at 500 hPa (contours) at analysis and three different forecast times. Dashes indicate the southern boundary of the region used for the computation of the spatial averages of Figs. 3 and 4.

  • View in gallery
    Fig. 9.

    As in Fig. 8, but for the CMC ensemble.

  • View in gallery
    Fig. 10.

    Spatial distribution of the mean uncertainty for the ECMWF ensemble. Shown are M (color shades) and the time mean of the geopotential analyses at 500 hPa (contours) at analysis and three different forecast times. Dashes indicate the southern boundary of the region used for the computation of the spatial averages of Fig. 3 and 4.

  • View in gallery
    Fig. 11.

    Spaghetti diagram for the ensemble of mean forecasts. The mean forecasts were obtained by averaging each member of the ECMWF ensemble over the investigated time period. Shown by gray contour lines are the 5350-gpm isohypses for the ensemble members. The black contour line shows the time mean of the ECMWF analyses for the investigated time period.

  • View in gallery
    Fig. 12.

    Spaghetti diagrams for select ensemble systems at the 360-h lead time. The mean forecasts were obtained by averaging each member of the ensemble over the investigated time period. Shown by gray contour lines are the 5350-gpm isohypses for the ensemble members. The black contour line shows the time mean of the ECMWF analyses for the investigated time period.

  • View in gallery
    Fig. 13.

    Spectral evolution of the forecast uncertainty and the ensemble spread for the meridional component of the wind vector at 500 hPa for the ECMWF and CMC ensembles. Shown are the meridional averages of the zonal power spectra of the meridional wind associated with (left) the ensemble spread and (right) the forecast uncertainty. The lowest curves show the spectra at analysis time, while the other curves show the spectra with 2-day increments of the forecast time (the top curves are for day-14 forecast time). The red curve shows the linear regression of the maximum power and the associated zonal wavenumber for all of the forecast times.

  • View in gallery
    Fig. 14.

    Lorenz curves for the forecast uncertainty () for the ECMWF, NCEP, CMC, and JMA ensembles.

  • View in gallery
    Fig. 15.

    Lorenz curves for the ensemble spread () for the ECMWF, NCEP, CMC, and JMA ensembles.

  • View in gallery
    Fig. 16.

    Estimates of as a function of () for the ECMWF, NCEP, CMC, and JMA ensembles.

  • View in gallery
    Fig. 17.

    Spatiotemporal evolution of (color shades) in the ECMWF ensemble forecasts. Also shown is the time mean of the geopotential analyses at 500 hPa (contours). Dashes indicate the southern boundary of the region used in the computation of the spatial averages shown in Figs. 3 and 4.

  • View in gallery
    Fig. 18.

    As in Fig. 17, but for the CMC ensemble.

  • View in gallery
    Fig. 19.

    The spatiotemporal evolution of the local ratio between and (color shades) in the ECMWF ensemble forecasts. Also shown is the time mean of the geopotential analyses at 500 hPa (contours). Dashes indicate the southern boundary of the region used in the computation of the spatial averages shown in Figs. 3 and 4.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 188 72 11
PDF Downloads 130 32 0

Forecast Uncertainty Dynamics in the THORPEX Interactive Grand Global Ensemble (TIGGE)

Michael A. HerreraTexas A&M University, College Station, Texas

Search for other papers by Michael A. Herrera in
Current site
Google Scholar
PubMed
Close
,
Istvan SzunyoghTexas A&M University, College Station, Texas

Search for other papers by Istvan Szunyogh in
Current site
Google Scholar
PubMed
Close
, and
Joseph TribbiaNational Center for Atmospheric Research, Boulder, Colorado

Search for other papers by Joseph Tribbia in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

This paper employs local linear, spatial spectral, and Lorenz curve–based diagnostics to investigate the dynamics of uncertainty in global numerical weather forecasts in the NH extratropics. The diagnostics are applied to ensembles in the THORPEX Interactive Grand Global Ensemble (TIGGE). The initial growth of uncertainty is found to be the fastest at the synoptic scales (zonal wavenumbers 7–9) most sensitive to baroclinic instability. At later forecast times, the saturation of uncertainties at the synoptic scales and the longer sustainable growth of uncertainty at the large scales lead to a gradual shift of the wavenumber of the dominant uncertainty toward zonal wavenumber 5. At the subsynoptic scales, errors saturate as predicted by Lorenz’s classic theory. While the ensembles capture the general characteristics of the uncertainty dynamics efficiently, there are locations where the predicted magnitude and structure of uncertainty have considerable time-mean errors. In addition, the magnitude of systematic errors in the prediction of the uncertainty increases with increasing forecast time. These growing systematic errors are dominated by errors in the prediction of low-frequency changes in the large-scale flow.

Corresponding author address: Michael A. Herrera, Department of Atmospheric Sciences, Texas A&M University, 3150 TAMU, College Station, TX 77843-3150. E-mail: mherr77m@gmail.com

Abstract

This paper employs local linear, spatial spectral, and Lorenz curve–based diagnostics to investigate the dynamics of uncertainty in global numerical weather forecasts in the NH extratropics. The diagnostics are applied to ensembles in the THORPEX Interactive Grand Global Ensemble (TIGGE). The initial growth of uncertainty is found to be the fastest at the synoptic scales (zonal wavenumbers 7–9) most sensitive to baroclinic instability. At later forecast times, the saturation of uncertainties at the synoptic scales and the longer sustainable growth of uncertainty at the large scales lead to a gradual shift of the wavenumber of the dominant uncertainty toward zonal wavenumber 5. At the subsynoptic scales, errors saturate as predicted by Lorenz’s classic theory. While the ensembles capture the general characteristics of the uncertainty dynamics efficiently, there are locations where the predicted magnitude and structure of uncertainty have considerable time-mean errors. In addition, the magnitude of systematic errors in the prediction of the uncertainty increases with increasing forecast time. These growing systematic errors are dominated by errors in the prediction of low-frequency changes in the large-scale flow.

Corresponding author address: Michael A. Herrera, Department of Atmospheric Sciences, Texas A&M University, 3150 TAMU, College Station, TX 77843-3150. E-mail: mherr77m@gmail.com

1. Introduction

Data assimilation produces the initial conditions of numerical weather forecasts by a statistical interpolation of the atmospheric observations (e.g., Szunyogh 2014). Thus, the initial conditions have an inherently random error component, which we will call the analysis uncertainty. The amplification of the analysis uncertainty by the chaotic dynamics of the atmosphere would lead to an inevitable growth of the magnitude of the forecast uncertainty, even if models were perfect replica of the atmosphere. Because models are not perfect and use statistical considerations to account for processes at the unresolved scales, model errors and model uncertainty also contribute to the forecast uncertainty.

In perhaps the most influential paper ever written on the dynamics of forecast error (uncertainty) growth, Lorenz (1969b) investigated the role of scale interactions in the error growth process. He argued that forecast errors saturated, predictability was lost, at increasingly larger scales as forecast time increased. Lorenz’s results were most recently revisited by Tribbia and Baumhefner (2004), Rotunno and Snyder (2008), and Durran and Gingrich (2014). Tribbia and Baumhefner (2004) augmented Lorenz’s description of the process by adding that in the extratropics the dominant errors asymptoted to the baroclinically active scales, where they then grew exponentially. Rotunno and Snyder (2008) replaced the original two-dimensional vorticity equation in Lorenz’s model by the surface geostrophic equation. They pointed out that the rapid downscale propagation of errors at the mesoscales played an important role in the rapid saturation of the forecast errors at the smaller scales. Durran and Gingrich (2014) extended this argument to emphasize that synoptic-scale errors, even if they had small magnitude, led to a rapid saturation of the errors at the smaller scales.

In their ensemble-based predictability studies, Kuhl et al. (2007) and Satterfield and Szunyogh (2010, 2011) (hereafter referred to collectively as KSS) observed the same spectral evolution of the forecast uncertainty as Tribbia and Baumhefner (2004). They also found that a 40–80-member ensemble of forecasts was efficient in capturing the dominant synoptic-scale patterns of forecast uncertainty. The present study extends the investigations of KSS to the THORPEX Interactive Grand Global Ensemble (TIGGE), which comprises operational global ensemble forecast data from the major operational centers. The primary objective of the paper is to verify that the findings of KSS also hold for the operational ensemble forecast systems. The results shown later strongly suggest the affirmative. The secondary objective of this paper is to provide information about the performance of the operational ensemble forecast systems.

The outline of the paper is the following. Section 2 provides background information about the TIGGE dataset, while section 3 describes the local diagnostics we employ. Section 4 describes the dynamics of the atmosphere for the investigated time period, while section 5 presents the results of our diagnostic calculations. Section 6 shows further diagnostic results for a select group of ensembles, and section 7 offers our conclusions.

2. The TIGGE dataset

We provide a general description of the TIGGE dataset and briefly discuss the techniques that the centers use to represent the effects of initial condition and model uncertainties.

a. The dataset

The TIGGE dataset is a collection of global ensemble forecasts from the major NWP centers (Bougeault et al. 2010; Swinbank et al. 2016). The goal of TIGGE is to provide ensemble data to support both academic research and operational product development. The forecasts are collected in real time and made available to the scientific community by the archiving centers in an easily accessible uniform format.

We analyze data from the following forecast centers: the European Centre for Medium-Range Weather Forecasts (ECMWF), the National Centers for Environmental Prediction (NCEP), the Met Office (UKMO), the China Meteorological Administration (CMA), the Japan Meteorological Agency (JMA), the Korean Meteorological Administration (KMA), the Meteorological Service of Canada (CMC), and Météo-France.

We do not consider data from two of the NWP centers, the Australian Bureau of Meteorology (BoM) and the Centro de Previsão de Tempo e Estudos Climáticos (CPTEC), which provide data to the TIGGE dataset: data were not available from the BoM ensemble for the time period of this study, while CPTEC discovered an error in their ensemble and was planning to regenerate the data.

b. Initial condition perturbations

The degrees of freedom of the dynamics of an operational model is orders of magnitude larger than the operationally attainable number of ensemble members. The forecast centers have addressed this challenging aspect of ensemble forecasting by developing techniques for the generation of initial condition perturbations that efficiently represent the growing part of the analysis (initial condition) errors. Table 1 shows a list of the ensemble generation techniques of the different centers.

Table 1.

Ensemble forecast systems included from TIGGE.

Table 1.

The bred vector method (Toth and Kalnay 1993, 1997), which was originally developed and implemented at NCEP, is currently used by CMA and KMA. To create the bred vectors, the analysis is randomly perturbed and the full nonlinear model is run for a short period (e.g., 6 h) for both the control (unperturbed) and perturbed analyses. The control forecast is subtracted from the perturbed forecasts and the resulting perturbations are rescaled to the magnitude of the initial perturbations. The cycle is repeated by adding the rescaled perturbations to the next analysis. After several days of “breeding,” growing patterns dominate the spatiotemporal evolution of the perturbations.

Another type of initial condition perturbations, which is used by ECMWF, JMA, and Météo-France are known as (right) singular vectors (Buizza et al. 1993; Molteni and Palmer 1993; Mureau et al. 1993); these vectors are the initial perturbations that grow fastest with respect to a preselected norm and optimization (forecast) time. For the ensembles included in this study, the norm is a quadratic norm with energy dimension (e.g., Buizza et al. 1993) and the optimization time is 48-h forecast hours. The Météo-France ensemble uses a combination of singular vectors and evolved singular vectors, where the evolved singular vectors are created such that the analysis time, for which the initial perturbations are created, coincides with the end of the optimization period. The evolved singular vectors are hoped to represent analysis uncertainties that were likely to grow in the analysis cycles of the immediate past. ECMWF also used evolved singular vectors in the past, but by the time of the present study they have switched to using an ensemble of data assimilations (EDA) to account for the error growth during the previous data assimilation cycles (Buizza et al. 2008). To create these perturbations, observations are perturbed randomly in accordance with their presumed error statistics in the data assimilation system; each set of perturbed observations is assimilated into a different ensemble member.

The method currently used for the generation of ensemble perturbations at NCEP is similar to the generation of bred vectors, but it uses information from the data assimilation system to determine a spatiotemporally varying rescaling factor. This method is called ensemble transform with rescaling (ETR) and was developed by Wei et al. (2008); ensemble perturbations valid for the analysis time are obtained through an ensemble transform of a previous set of forecast perturbations, taking into account the observation error statistics and centering the perturbations on the analysis.

The Met Office uses a local ensemble transform Kalman filter (ETKF) to generate their perturbations (Bishop et al. 2001; Wang and Bishop 2003; Bowler and Mylne 2009). The largest difference between the ETKF and the ETR methods is that the ETKF produces an ensemble of full analyses rather than an ensemble of rescaled perturbations. The analysis perturbations, which are obtained by taking the difference between the members of the analysis ensemble and the mean of the analysis ensemble, are added to the operational 4D-Var analysis to obtain the ensemble of perturbed initial conditions. Last, CMC uses an ensemble Kalman filter (EnKF) to generate the analysis ensemble. Unlike at UKMO, their ensemble is centered on the mean analysis produced by the EnKF.

c. Model error parameterization techniques

In addition to chaotic model dynamics acting on uncertain initial conditions, model errors also contribute to the forecast error growth. Model errors affect the forecasts continuously during the entire forecast period. They also contribute to the initial conditions uncertainty through the forecast phase of the analysis cycles.

The main sources of model errors are thought to be the parameterization schemes for the subgrid processes. One technique to account for these sources is the method of stochastically perturbed parameterization tendencies (SPPT; Buizza et al. 1999; Palmer et al. 2009). This technique perturbs the total contribution of the parameterized processes to the tendency of the state variables in the model. Another technique is to use different parameterization schemes for the same processes, or to use different values of the prescribed parameters of the parameterization schemes. This approach is known as the multiphysics technique (Berner et al. 2011; Houtekamer 2002).

The effects of uncertainties injected at the smallest resolved scales cannot be directly simulated by the models, because the interactions between those scales and the larger scales are distorted by the models: some scale interactions are explicitly eliminated by the truncation strategies, while others are eliminated by dampening the smaller-scale motions. Time integration schemes also contribute to the diffusiveness of the models at scales where nature is not diffusive. An approach to make the representation of the effect of upscale propagating uncertainties by the ensemble more realistic, called stochastic energy backscattering (SKEB), was introduced and described by Shutts (2005, 2013), Berner et al. (2009), Bowler et al. (2009), Charron et al. (2010), and Tennant et al. (2011). One other method used to account for this uncertainty is called stochastic time tendency perturbations (STTP; Hou et al. 2008). Whereas SKEB focuses on subgrid-scale error, STTP adds stochastic forcing at all scales. Since SPPT/multiphysics and SKEB/STTP simulate different aspects of model error dynamics, they can be used in conjunction. This practice is followed at both ECMWF and CMC.

3. Local diagnostics

An ensemble forecast provides a flow (synoptic situation) dependent prediction of the probability distribution of the forecast uncertainty. We focus on examining the mean and the covariance matrix of the predicted probability distribution. The significance of the covariance matrix of the distribution is that it describes both the structure and the magnitude of the predicted uncertain flow features. In addition, under the assumption that the probability distribution of the uncertainty is Gaussian, the mean and the covariance matrix together provide a complete description of the predicted probability distribution.

a. Local vectors and their covariance

Following the approach of KSS, we define a local state vector to describe the state in a local atmospheric volume centered at model gridpoint . The components of are the gridpoint variables of the model in . We assume the availability of a K-member forecast ensemble and define the K-member ensemble of local state vectors , k = 1, …, K, by the relevant components of the state vectors that represent the ensemble. Then, a K-member ensemble of local perturbations, , k = 1, …, K, can be defined by
e1
where
e2
is the local ensemble mean, which is the prediction of the mean of the probability distribution of the local forecast uncertainty. We note that the same local framework is employed in the widely used local ensemble transform Kalman filter (LETKF) data assimilation scheme (Ott et al. 2004; Hunt et al. 2007; Szunyogh et al. 2005, 2008).
In what follows, we treat all local vectors as column vectors. The prediction of the local covariance matrix of the forecast uncertainty is
e3
The local state vector, the local ensemble perturbations, and the local covariance matrix can be defined for all locations , local volumes , and forecast times , (including the analysis time, ). We assume that the spaces spanned by the local ensemble perturbations are linear spaces. To be precise, we define the space of local ensemble perturbations by the range of , and assume that any linear combination of the local ensemble perturbations is a plausible local perturbation of the atmospheric state. Because we compute the diagnostics for all locations and use a location-independent definition of the local volumes , we drop the subscripts from the notation. To further simplify notation, we also drop the argument . For instance, we replace the notation by .

b. Diagnostics for the predicted magnitude of the uncertainty

1) Optimality conditions

Our error diagnostics are based on investigating the statistical properties of the difference between a proxy of the true state and the ensemble mean :
e4
We treat the values of computed for the different forecasts, forecast times, and locations as realizations of the random vector variable . In ensemble forecasting, the difference is usually interpreted as an estimate of the error in the ensemble mean forecast. This terminology is fully justified when the ensemble mean is used as a deterministic forecast of the atmospheric state. In our interpretation, is an estimate of the difference between the (unknown) true state and the (unknown) true mean of the probability distribution of the state given all sources of uncertainty:
e5

The vector is a representation of the forecast uncertainty, because if there were no uncertainties, would be identical to , leading to . As the magnitude of the forecast uncertainty increases with increasing forecast time, becomes increasingly different from , leading to an increase of the magnitude of . At the forecast time at which predictability is completely lost, becomes identical to the climatological mean state of the atmosphere and the magnitude of converges to the magnitude of the climatological atmospheric variability (Epstein 1969; Leith 1974).

Because
e6
where is the error in the proxy for the true state and
e7
where is the error in the prediction of the mean, Eq. (5) can be also written as
e8
Equation (8) shows that the estimate of has an error of . Similar to , the random variables , , and depend on the location, the initial time of the forecast, and the forecast time, respectively.
According to its definition, the random variable satisfies the condition that
e9
where is the expected value function. Hence, the estimate of should satisfy the following condition:
e10
In addition, because is considered the prediction of the covariance matrix of in the ensemble prediction, the two covariance matrices should also be equal. Because these two conditions cannot be verified for a single realization of , ensemble forecast verification techniques investigate whether the behavior of the ensemble forecast system is consistent or not with these two conditions over a large number of realizations of .

2) The magnitude of the forecast uncertainty

The mean square magnitude of the ensemble-based estimate of the difference between the true state and the true mean of the probability distribution of the state is
e11
A standard approach for the evaluation of an ensemble forecast system is to verify whether the mean of the ensemble variance
e12
satisfies the approximate equality
e13
This criterion is a necessary condition for being an accurate prediction of (the trace of the covariance matrix of ) under the assumption that .

3) The uncertainty in the proxy for the true state

The presence of is a limitation of any forecast verification technique. In the particular case of , the contribution of is not negligible at short (e.g., shorter than 12–48 h) forecast times, at which its magnitude can be comparable to the magnitude of . The correlations between and the error components and can be reduced to near zero, even at short forecast times, by the proper choice of . For instance, when a time series of is defined by analyses, those analyses should be other than those used for the production of the verified forecasts.

4) The error in the prediction of the mean

A nonzero value of is usually expected due to the inevitable presence of systematic model errors that can lead to a drift of the predicted probability distributions in state space. In addition, flaws in the design of the ensemble system can also contribute to the error in the ensemble mean. A testable sufficient condition for at the longer forecast times is . The sufficient nature of this condition can be seen by first making use of Eqs. (8) and (9), which lead to
e14
by taking into account that the magnitude of is small and its contribution to can be neglected if h.
Because
e15
where is the variance function for the time period of verification, can contribute to . If the climate and the model representation of the climate were both stationary, for an infinitely long forecast, M would be equal to the mean error in the climatological mean state of the model. However, because forecast verification is usually done only for a season of a single year and numerical predictions are for finite forecast times, M typically asymptotes to a value that includes systematic errors in the prediction of the lower-frequency variability of the atmosphere.

5) Lorenz curves

The evolution of , , and M for the different TIGGE ensembles will be compared by figures whose format will be mostly familiar to the reader. The only unusual comparison we do is based on fitting Lorenz curves (Lorenz 1969a, 1982) to both and , and comparing the parameters of the fitted curves. We parameterize the Lorenz curves by the following function:
e16
which was proposed by Dalcher and Kalnay (1987). Our choice of this particular parameterization is motivated by the paper Magnusson and Kallen (2013), which used it successfully to separate the factors that contributed to the improvement of the operational ECMWF forecasts from 1979 to 2011.

In our application of Eq. (16), F is either or , α is the parameter that describes the (exponential) growth of F for the linear phase of uncertainty dynamics, and is the saturation (asymptotic) value of F at long forecast times. For , the parameter β can be considered a static (time independent) estimate of the contribution of model errors to the forecast uncertainty tendency, while for , it can be considered an estimate of the contribution of the technique used for the representation of the effect of model errors to the tendency of . If β is smaller for than , the chosen technique(s) overestimates the contribution of model errors, while if it is smaller for than , it underestimates the contribution of model errors.

We emphasize that Eq. (16) is a crude parameterization of the function that describes the error growth process. It is based on the assumption that the initial error growth process is linear and nonlinear effects become important only later, once the magnitude of the errors becomes sufficiently large. While this is a reasonable assumption for the error growth at the synoptic scales, it is clearly violated by the rapidly saturating errors at the smaller scale. In addition, earlier studies (e.g., Orrell et al. 2001; Vannitsem and Toth 2002; Nicolis et al. 2009) also demonstrated that model errors tended to lead to a nonlinear short-term error growth. Hence, Eq. (16) is expected to provide a better description of the error growth process from the forecast times at which synoptic-scale errors become dominant; and the estimates of β should be considered a particularly crude estimate.

c. Diagnostics for the predicted structure of the uncertainties

The local vector can be decomposed as
e17
where is the component of that projects onto and is the component that projects onto the null space of . Heuristically, the vector represents the collection of uncertain local forecast features that the ensemble is able to capture. Likewise, represents the collection of uncertain local forecast features that the ensemble is unable to capture. (This interpretation assumes that .)
The set of normalized eigenvectors associated with the largest eigenvalues of provide a convenient orthonormal basis to compute by
e18
The origin of the local orthogonal coordinate system defined by the basis vectors is the local ensemble mean . Heuristically, these basis vectors describe the structure of the local uncertain forecast features.
The efficiency of in capturing can be assessed by comparing
e19
to . always satisfies the relation , with the equality indicating the ideal situation, in which lies entirely in . There are three reasons why can be smaller than its optimal value of . First and most importantly, may not provide a perfect representation of the space in which evolves. Second, the origin may be shifted, that is, . Third, typically has no significant projection on , which reduces at short forecast lead times.

Finally, it should be noted that there are approaches different than ours to define an orthogonal basis for the investigation of the evolution of ensemble perturbations. For instance, Leutbecher and Lang (2014) defined an orthogonal basis by the leading right singular vectors of the tangent linear version of the model, while Zagar et al. (2015) defined an orthogonal basis by the normal mode functions consisting of vertical structure functions, each associated with a set of horizontal Hough functions.

d. Estimation of the expected value

We estimate the expected value by either an average over all forecasts of the same lead time and all locations in the verification region, or an average over all forecasts of the same lead time. In the former case, the result is a scalar that depends only on the forecast lead time, while in the latter case, the result is a field of gridpoint values that depends on the forecast lead time.

We compute diagnostics for forecasts that were started between 0000 UTC 1 January and 1800 UTC 29 February 2012. The diagnostics are computed for the entire forecast range of each ensemble system. Diagnostics that require the estimation of temporal means are computed by taking averages over all forecasts of equal forecast time. Spatiotemporal means for the NH extratropics are computed by averaging the temporal means over all locations between 30° and 75°N.

4. Experiment design

The local volume is defined by the atmospheric column given by 5 × 5 horizontal grid points centered at , and the pressure levels between 1000 and 200 hPa. Because the dataset has a 2.5° × 2.5° horizontal resolution, the horizontal dimension of a local volume in the midlatitudes is about 1000 km × 1000 km. Components of the local state vector are defined by the virtual temperature, and zonal and meridional wind gridpoint variables. The different components are scaled such that the Euclidean norm of the local state vector has the dimension of energy (Talagrand 1981; Buizza et al. 1993).

We use ECMWF analyses as the proxy for the true state except for the computations of the diagnostics for the ECMWF and Météo-France ensembles, in which NCEP analyses are used as . These choices for are made to ensure that is statistically independent of and . To test the sensitivity of the results to the choice of the verifying data, diagnostic calculations for additional ensembles were also carried out by using the NCEP analyses as the proxy of the true states.

5. The atmospheric flow

For the time period of our investigation, we describe the synoptic-scale transients with the help of the eddy kinetic energy equation (EKE) and the low-frequency transients by the zonal anomalies of the seasonal mean flow.

High-frequency (synoptic scale) transient components of the flow

The EKE equation of Orlanski and Katzfey (1991), also see Orlanski and Chang (1993) and Chang (2000), is
e20
In this equation, the prime indicates the eddy component of the state variables and is the eddy kinetic energy given by
e21
where is the eddy component of the horizontal wind vector. The symbols , , , and denote the horizontal nabla operator, the three-dimensional nabla operator, the mean component of the horizontal wind vector, and the eddy component of the three-dimensional wind vector for pressure vertical coordinate, respectively. Otherwise, the conventional notation is used for the state variables. The bar denotes a seasonal mean, while the symbol indicates a vertical average in pressure coordinate system, and indicates a surface integral across the surface (s) or top (t) of the model atmosphere.

The first term of the right-hand side describes the horizontal eddy kinetic energy transport, the second term is the geopotential flux convergence, the third term is the baroclinic energy conversion, and the fourth term is the barotropic energy conversion. Term five describes the vertical eddy kinetic energy transport through the bottom and the top surfaces, while term six represents the transport of eddy potential energy through the same surfaces. Finally, the last term is the residue term that represents the bulk effect of the errors of the numerical calculations and all processes unaccounted for by the other terms. The most important such process is dissipation, which usually makes the residue term negative. Because not all variables necessary for the computation of the terms of the eddy kinetic energy equation are available in the TIGGE dataset, we use data from the ERA-Interim reanalysis for the description of the flow. Unlike the previous diagnostics that are calculated on local volumes, the eddy kinetic energy equation is calculated on the global grid.

The computation of the terms of the eddy kinetic energy equation starts with a decomposition of the spatiotemporally evolving atmospheric state variables into a spatially varying time-mean component and a spatiotemporally evolving eddy component. We compute the time mean for January–March, because even though all forecasts start in January and February, some of them end in March. The time-mean component of the geopotential height field at the 500-Pa pressure level is shown in Fig. 1 for both the ECMWF and NCEP analyses: the time-mean flow has a dominantly zonal wavenumber-2 structure, with negative zonal anomalies in the Pacific and the Atlantic storm-track regions and positive zonal anomalies in the exit regions of the storm tracks.

Fig. 1.
Fig. 1.

Zonal anomalies of the time-mean flow. Color shades indicate the zonal anomalies for the investigated time period based on the (left) ECMWF analyses and (right) NCEP analyses, while contours show the time-mean flow (geopotential height) at the 500-hPa level based on the same analyses.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

Figure 2 shows the time-mean eddy kinetic energy (top-left panel) and the time mean of the three terms of the eddy kinetic energy that dominate the changes in the time-mean eddy kinetic energy (other three panels). These terms represent baroclinic energy conversion (top-right panel), barotropic energy conversion (bottom-left panel), and the horizontal transport of the eddy kinetic energy (bottom-right panel). The largest local maxima of the eddy kinetic energy are located in the eastern sector of the Pacific storm track. These maxima are due to the local generation of kinetic energy by baroclinic energy conversion and the transport of eddy kinetic energy generated upstream by baroclinic energy conversion.

Fig. 2.
Fig. 2.

The time mean of the eddy kinetic energy conversion processes for the investigated time period. Color shades show the time mean of the (top left) eddy kinetic energy (J), (top right) baroclinic energy conversion (J day−1), (bottom left) barotropic energy conversion (J day−1), and (bottom right) horizontal transport of the eddy kinetic energy (J day−1).

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

Barotropic energy conversion is the (nonlinear) transfer of kinetic energy between the synoptic-scale transients and the seasonal mean flow. Where it is positive, kinetic energy is transferred to the synoptic-scale eddies, while where it is negative, kinetic energy is transferred to the seasonal mean flow. While kinetic energy is transferred to the seasonal mean flow in the exit regions of the storm tracks, kinetic energy is transferred to the synoptic-scale eddies over North America and western Europe.

A comparison of Figs. 2 and 1 show the close relationship between the high- and low-frequency transients: baroclinic energy conversion at the synoptic scales is the most intense in regions of the negative anomalies of the time-mean flow, while (negative) barotropic energy conversion from the synoptic to the large scales plays a direct role in the slow changes of the large-scale flow in regions of positive anomalies. These nonlinear barotropic energy conversion processes control the energy transfer between the high-frequency transients and the slowly varying large-scale flow.

6. Results on the predictions of the magnitude of the uncertainty

In this section, our attention is focused on studying the relationship between the evolutions of , , and M in the forecasts. We examine the behavior of both the spatiotemporally and the temporally averaged forms of the three diagnostics.

a. The evolution of , , and in the forecasts

1) Diagnostics based on averages over all forecasts and locations

We first examine the evolutions of , , and in the forecasts qualitatively, with the help of Figs. 3 and 4, which show the evolution of the spatiotemporally averaged version of the three quantities. A common feature of the behavior of the different ensembles at analysis time is that (green curve) tends to be smaller than (black curve). That is, the ensembles have a tendency to underestimate the analysis uncertainty. This feature is the most pronounced for the JMA ensemble and barely noticeable for the CMC ensemble. Because the match between and for the latter ensemble is essentially perfect at all forecast times between 12 and 132 h, the slight difference at analysis time is likely to be due to uncertainty in the proxy for the true state rather than to an underestimation of the magnitude of the initial uncertainty.

Fig. 3.
Fig. 3.

The evolution of the diagnostics—, , , and —in the forecasts for four of the ensembles averaged over the NH extratropics and all forecasts started between 1 Jan and 29 Feb 2012. It should be noted that the models have different max forecast lead times.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

Fig. 4.
Fig. 4.

The evolution of the diagnostics—VS, TV, TVS and —in the forecasts for the remaining four ensembles averaged over the Northern Hemisphere extratropics and all forecasts started between 1 Jan and 29 Feb 2012. It should be noted that the models have different max forecast lead times.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

For most ensembles, quickly (in about 48–72 forecast hours) asymptotes to . The rapid recovery of the ensemble variance is particularly notable for the JMA ensemble. The unique short-term behavior of this ensemble can be explained by the fact that this is the only ensemble in TIGGE that is purely based on right singular vector initial condition perturbations: because the right singular vectors grow very rapidly during the optimization period, which is 48 h for the JMA ensemble, the magnitude of the analysis perturbations must be small to avoid overshooting at the 48-h forecast time. The ensemble that shows a somewhat similar behavior, but with a much less severe underestimation of the analysis uncertainty, is the ECMWF ensemble. The similarity is not by accident; some of the initial condition perturbations in the ECMWF ensemble are right singular vectors. The underestimation of the uncertainty in the ECMWF ensemble is much less severe, because it mixes the right singular vectors with perturbations produced by an ensemble of data assimilations. The latter perturbations grow much slower than the right singular vectors, but their initial magnitude is larger, leading to an overall larger magnitude of the analysis perturbations.

The ensemble for which the gap between and remains relatively large at all forecast times is the CMA ensemble. This behavior is most likely due to the feature of the CMA ensemble that it is one of only two ensembles in TIGGE that does not use any “parameterization” scheme to continuously increase the magnitude of the evolving forecast perturbations. The only other TIGGE member that does not “parameterize” the effects of model uncertainty is the KMA ensemble, but for that ensemble the gap between and is smaller than for the CMA ensemble at initial time, which helps at the longer forecast times as well.

While there are more pronounced differences in the evolution of (purple curve) than in the evolution of and between the different ensembles, there are also some important similarities: the relative contribution of to is the largest at analysis time, while is typically an order of magnitude smaller than . The shape of the curves that describe the evolution of suggests that is growing due to systematic errors in the prediction of the low-frequency transients.

2) Sensitivity of the results to the choice of the proxy for the true state

To test the robustness of the diagnostic results shown in Figs. 3 and 4 to the choice of the proxy for the true state, we computed some of the diagnostics using analyses from different centers for the definition of . An example for the results of these calculations is shown in Fig. 5, which shows the diagnostics for the UKMO ensemble using ECMWF or NCEP analyses as . While the results slightly change quantitatively,1 the choice of has no effect on our qualitative observations about the relationships between the evolution of the diagnostics.

Fig. 5.
Fig. 5.

Illustration of the robustness of the results of Figs. 3 and 4 to the choice of the proxy for the true state. The diagnostics are shown for the UKMO ensemble for the cases in which the proxy for the true state is defined by (left) the ECMWF analyses and (right) the NCEP analyses.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

3) Diagnostics based on averages over all forecasts

To save space, we show temporally averaged forms of the diagnostics only for selected ensembles. Figures 6 and 7 show at four different forecast times for the ECMWF and the CMC ensembles, respectively. We choose these two ensembles, because while they are among the better performing ensembles, they are generated by using different techniques for the generation of the initial conditions and the representation of the effects of model errors. As discussed later, the results of our investigation also suggest that ECMWF and CMC use different tuning conditions for their systems. The two figures show that the differences between the two systems are the largest at analysis time and rapidly diminishing with increasing forecast time. These diminishing differences are the result of the fast growth of TV in the storm-track regions in both ensembles. By forecast time 360 h, the forecast uncertainty becomes the largest in the exit region of the Pacific and the Atlantic storm tracks, where the magnitude of the (positive) zonal anomaly of the time-mean flow is the largest (see Fig. 1). This behavior suggests that uncertainties in the prediction of both the high- and low-frequency transients contribute to the large forecast uncertainty. As for the differences at analysis time, the large differences in the region of the Tibetan Plateau and the Himalayas are most likely due to differences in the orography of the verified and verification datasets. The related local maximum quickly disappears with increasing forecast time for both ensembles. The more important initial difference is the markedly lower magnitude of the uncertainty in the storm-track regions, especially over the Pacific, for the CMC ensemble. This initial difference is most likely due to the larger magnitude of the CMC analysis perturbations.

Fig. 6.
Fig. 6.

Spatial distribution of the average forecast uncertainty for the ECMWF ensemble. Shown are (color shades) and the time mean of the geopotential analyses at 500 hPa (contours) at analysis and three different forecast times. Dashes indicate the southern boundary of the region used for the computation of the spatial averages of Figs. 3 and 4.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

Fig. 7.
Fig. 7.

As in Fig. 6, but for the CMC ensemble.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

Figures 8 and 9 show at four different forecast times for the ECMWF and the CMC ensembles, respectively. A striking feature of the two figures is their similarity. In particular, the growth of the ensemble is the fastest in the storm-track regions. Both ensembles clearly underestimate the analysis uncertainty, but the underestimation is more severe for the ECMWF ensemble. For the CMC ensemble, slightly underestimates in the storm-track region at all forecast times, but, in general, it correctly captures the main patterns of uncertainty. For the ECMWF analysis, slightly underestimates at all locations at the 120-h forecast time, but later there are an increasing number of locations where it overestimates TV. The most important shortcoming of for the ECMWF ensemble is that at the 360-h lead time, its maximum of the Atlantic is shifted westward (from Iceland to Newfoundland) compared to the related maximum of .

Fig. 8.
Fig. 8.

Spatial distribution of the average ensemble spread for the ECMWF ensemble. Shown are (color shades) and the time mean of the geopotential analyses at 500 hPa (contours) at analysis and three different forecast times. Dashes indicate the southern boundary of the region used for the computation of the spatial averages of Figs. 3 and 4.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

Fig. 9.
Fig. 9.

As in Fig. 8, but for the CMC ensemble.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

To shed some light on the origin of the aforementioned shift, we also plot the evolution of M for the ECMWF ensemble (Fig. 10). The growing component of M, which becomes dominant by forecast time 360 h, is associated with errors in the prediction of the zonal anomalies of the time mean-flow, which are shown in Fig. 1. In particular, the dipole pattern in the Atlantic region in the lower-right panel of Fig. 10 indicates a significant error in the prediction of the large-scale flow in that region. The fact that the growing component of M is due to growing systematic errors in the prediction of the large-scale flow is further illustrated by Fig. 11. This figure takes advantage of the property of M that it can be written as
e22
The figure shows (black lines) and the K-member ensemble of (gray lines). The variable M would be zero, if the mean of the gray curves was identical with the black curve. This ideal situation cannot occur, because the ensemble members do not capture the zonal anomalies of the large-scale flow. Figure 12, which shows the same type of spaghetti diagram for four different ensembles at the 360-h forecast time, illustrates that the behavior we have just described is a common shortcoming of all ensemble systems.
Fig. 10.
Fig. 10.

Spatial distribution of the mean uncertainty for the ECMWF ensemble. Shown are M (color shades) and the time mean of the geopotential analyses at 500 hPa (contours) at analysis and three different forecast times. Dashes indicate the southern boundary of the region used for the computation of the spatial averages of Fig. 3 and 4.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

Fig. 11.
Fig. 11.

Spaghetti diagram for the ensemble of mean forecasts. The mean forecasts were obtained by averaging each member of the ECMWF ensemble over the investigated time period. Shown by gray contour lines are the 5350-gpm isohypses for the ensemble members. The black contour line shows the time mean of the ECMWF analyses for the investigated time period.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

Fig. 12.
Fig. 12.

Spaghetti diagrams for select ensemble systems at the 360-h lead time. The mean forecasts were obtained by averaging each member of the ensemble over the investigated time period. Shown by gray contour lines are the 5350-gpm isohypses for the ensemble members. The black contour line shows the time mean of the ECMWF analyses for the investigated time period.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

b. Spectral evolution of and in the forecasts

We illustrate the spectral evolution of the forecast uncertainty and the ensemble spread with the example of the ECMWF and the CMC ensembles. Figure 13 shows the spectral evolution of the two quantities for the meridional component of the wind vector at 500 hPa. The left panels show the evolution of the spectral distribution of , and the right panels show the evolution of the spectral distribution of for the meridional wind vector component of the state vector. This figure was obtained by first computing the zonal power spectra at each latitude within the verification region, then computing the meridional average of the zonal spectra.

Fig. 13.
Fig. 13.

Spectral evolution of the forecast uncertainty and the ensemble spread for the meridional component of the wind vector at 500 hPa for the ECMWF and CMC ensembles. Shown are the meridional averages of the zonal power spectra of the meridional wind associated with (left) the ensemble spread and (right) the forecast uncertainty. The lowest curves show the spectra at analysis time, while the other curves show the spectra with 2-day increments of the forecast time (the top curves are for day-14 forecast time). The red curve shows the linear regression of the maximum power and the associated zonal wavenumber for all of the forecast times.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

The evolution of the spectra of (right panels) is very similar for the two ensembles. At analysis time, the two spectra are white, except for a slow drop of the power at the highest wavenumbers. Initially, the growth is the fastest at wavenumber 9, but with increasing forecast time, the maximum power gradually shifts to wavenumber 4 by forecast time day 14. This shift of the maximum power toward the lower wavenumbers is the result of the earlier saturation of the power at the wavenumbers at which it grows faster initially. At the subsynoptic scales (wavenumbers larger than about 12–14) the errors saturate as Lorenz (1969b) described: the steepness of the saturation spectra is , along which the uncertainty and the spread saturate at increasingly larger scales as forecast time increases.

The difference in the spectra of the ensemble spread between the two ensembles (left panels) is the largest at analysis time: the shape of the spectra of for the ECMWF ensemble is similar to that of the spectra of , but it has significantly less power; while the shape of the spectra of for the CMC ensemble is different from that of the spectra of , but the average power of the and spectra are more similar than for the ECMWF ensemble. The evolution of captures the generic characteristics of the evolution of for both ensembles. An interesting difference, however, is that at forecast time day 14, the power has its maximum at wavenumber 5 rather than wavenumber 4 for both ensembles.

c. Qualitative description of the forecast uncertainty growth process

The information provided by Figs. 13, 6, 7, and 10 suggests the following general description of the forecast uncertainty growth process in the NH extratropics:

  1. the initial growth is the fastest at the synoptic scales that are most sensitive to baroclinic instability;

  2. the growing, and later saturating, synoptic-scale features of uncertainty fill the region that extends from the entrance region of the Pacific storm track to the exit region of the Atlantic storm track;

  3. because the position and the spatial structure of the storm-track regions are controlled by the slowly varying large-scale flow, the low-frequency transients have a major influence on the spatiotemporal distribution of the forecast uncertainty;

  4. as uncertainties start saturating at the scales most sensitive to baroclinic instability, the wavenumber of dominant instability gradually shifts toward the larger scales (lower wavenumbers); and

  5. at the subsynoptic scales (zonal wavenumbers larger than about 12–14), the uncertainty saturates as predicted by Lorenz’s theory.

The diagnostic results for the ensemble spread (Figs. 8, 9, 10, and 13) suggest that all ensemble forecast systems can capture the main characteristics of the error growth process. Figures 3 and 4 show, however, that the differences between the models, analysis systems and ensemble generation techniques have important effects on the accuracy of the quantitative prediction of the uncertainty.

d. Lorenz curve–based analysis of the evolution of and with increasing forecast time

1) Estimation of the parameters

Obtaining estimates of the parameters of the Lorenz curve requires the availability of at each forecast time. We compute an approximate value of by the centered-difference scheme:
e23
where 12 h. We compute for all forecast times h, but we ignore the first ( h) data point when fitting the Lorenz curve for . The reason to exclude this data point is that the estimates of both and F have a large relative error due to the error in the verification data.
The parameters α, β, and can be estimated by fitting a second-order polynomial to the pairs of data using the standard least squares approach for function fitting. The estimates of α and β are
e24
e25
where is the coefficient of the second-order term of the fitted polynomial and is the coefficient of the zeroth-order term, while is the positive root of the fitted polynomial.

2) Comparison of the estimated parameters of the Lorenz curves

The estimated parameters of the Lorenz curves for the forecast uncertainty () and the ensemble spread () are summarized by Table 2. In addition, the pairs of values and the fitted curves for four selected ensembles (ECMWF, NCEP, CMC, and JMA) are shown by Fig. 14 for the forecast uncertainty, and by Fig. 15 for the ensemble spread. The two figures show that there are some outliers in terms of the quality of the fit of the curves to the data. In particular, in the upper-left panel of Fig. 14 the forecast error is significantly overestimated at both 12- and 24-h forecast times. Hence, in addition to the data point for 12-h forecast time, the data point for 24-h forecast time is also excluded from the estimation of the parameters of the Lorenz curve. The bottom-right panel of Fig. 15 shows that the curve fitting for the ensemble spread has also failed. This failure suggests that the rapid initial growth of the SV perturbations is inconsistent with the growth process that a Lorenz curve can describe. The results for the CMC ensemble (bottom-left panel) are also somewhat suspect.

Table 2.

Estimates of the parameters of the Lorenz curves for the different ensembles. The crisscross (×) indicates parameters for which the estimation process failed, while italics indicate estimates that most likely have unusually large errors.

Table 2.
Fig. 14.
Fig. 14.

Lorenz curves for the forecast uncertainty () for the ECMWF, NCEP, CMC, and JMA ensembles.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

Fig. 15.
Fig. 15.

Lorenz curves for the ensemble spread () for the ECMWF, NCEP, CMC, and JMA ensembles.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

To make the interpretation of the values of α in Table 2 more transparent, the table also includes the values of for day. This quantity is the daily growth rate of F for the time range in which the uncertainty dynamics is linear to a good approximation. For the ECMWF, NCEP, UKMO, and KMA ensembles, the daily linear growth rate for the forecast uncertainty is 1.5. The same growth rate for the ensemble spread is slightly lower (1.4) for the ECMWF, UKMO, and KMA ensembles, and slightly higher (1.6) for the NCEP ensemble. The latter result suggests that the underestimation of by in the NCEP ensemble (top-right panel of Fig. 3) is not due to an underestimation of the linear error growth. The CMC and the CMA ensembles underestimate the linear error growth to various degrees. (As indicated earlier, the estimation of the parameters for has failed for the JMA ensemble.)

We recall from section 3b(5) that we consider the parameter β to be a measure of the contribution of model errors to the growth of the forecast uncertainty , and a measure of the contribution of the parameterization of model errors to the ensemble spread . (The model error parameterization techniques used by the ensembles of the different centers are listed in Table 1.) Examining the results for β, it should always be kept in mind that the estimates are particularly sensitive to the errors of curve fitting at the short forecast times.

The most interesting conclusions that can be drawn about β are the following. An inefficient representation of the model error forcing by the ensemble is the most likely main source of the underestimation of the forecast uncertainty by the NCEP ensemble (top-right panel of Fig. 3). The good balance between and in the CMC ensemble (bottom-left panel of Fig. 3) is the result of a compensation of the underestimation of the linear error growth by an overestimation of the model error forcing. We note that CMC made major changes to their ensemble system, including the representation of model error forcing, in 2013 (recall that our study is for 2012 data). The results of Reynolds et al. (2015) suggest that those changes led to significant changes in the behavior of the CMC ensemble. It is highly likely, therefore, that the Lorenz curves would behave differently than reported here for the current operational configuration of the CMC ensemble. The estimate of is an estimate of the saturation level of the forecast uncertainty. All ensembles, except for the ECMWF ensemble, underestimate this saturation level ().

For comparison, we also computed the parameters of Lorenz curves for the more conventional choice of the 500-hPa geopotential height rather than a combination of the virtual temperature and the two horizontal wind components in the layer between the 1000- and 200-hPa levels. In these calculations, we computed and without localization. (The expected value was estimated by temporal averaging only.) Because a norm based on the geopotential height gives much less weight to errors at the smaller scales than a norm based on energy, the parameters of these Lorenz curves are far less sensitive to error growth at the smaller scales than those that we described earlier. As expected, we found the curve fitting more robust for the geopotential height. In particular, we did not have to exclude data points at the short forecast times and the curve fitting never failed. The results are summarized in Table 3. The linear error growth is lower than before (1.2 rather than 1.5–1.6) and uniform for the different ensembles. All ensembles do a good job at capturing this linear growth rate, which suggests that they are all tuned to perform well for diagnostics based on the 500-hPa geopotential height. The ECMWF ensemble also correctly simulates the contribution of model errors to the forecast uncertainty β, while the NCEP ensemble still underestimates the contribution of model errors to the forecast uncertainty. Interestingly, the new Lorenz curves suggest that the CMC ensemble greatly underestimates the contribution of model errors, while the earlier curves indicated that it greatly overestimated the contribution of model errors. This discrepancy suggests that the CMC ensemble introduces the effect of model errors at the wrong scales and/or model levels. We suspect that the changes made to the CMC system in 2013 greatly reduced this discrepancy.

Table 3.

Estimates of the parameters of the Lorenz curves for the different ensembles using 500-hPa geopotential height without localization in the calculation of and .

Table 3.

3) The relationship between and

We also prepared Lorenz curve–style figures for (Fig. 16), but without fitting curves to the pairs of data points. These figures are evidently more dissimilar for the four ensembles than those for and . This difference is most likely due to the fact that model errors can be diverse, while the evolution of and is primarily driven by the universal sensitivity of the synoptic-scale transients to random perturbations. In general, the shape of the curves is consistent with our earlier conclusion that the growth of is dominated by errors in the prediction of low-frequency transients.

Fig. 16.
Fig. 16.

Estimates of as a function of () for the ECMWF, NCEP, CMC, and JMA ensembles.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

7. Results on predictions of the structure of uncertainty

We examine the evolution of in the forecasts by comparing it to the evolution of and .

a. Diagnostics based on averages over all forecasts and locations

For the examination of the spatiotemporally averaged form of , we return to Figs. 3 and 4. While comparing the evolution of and , it should be kept in mind, that the two diagnostics must satisfy the relation at all forecast times. In addition, smaller differences between and indicate a better performance of the ensemble in capturing the local forecast uncertainty.

The generally small differences between (red curves) and (black curves) beyond the 48–72-h forecast times for the TIGGE ensembles is in a good agreement with the behavior that was reported by KSS for a research ensemble. This result confirms that an ensemble that is operationally attainable in size can efficiently span the local linear space of the forecast uncertainty beyond forecast times 48–72 h.

For most ensembles, the asymptotic value of tends to a level that is lower than the saturation level of . The only ensemble that is virtually unaffected by this problem is the ECMWF ensemble, while the ensemble that it affects most severely is the CMA ensemble. This result for the ECMWF analysis suggests that it captures all important forecast uncertainties that develop during the investigated time period.

A small difference between and indicates that the ensemble captures the important uncertain forecast features, but it does not guarantee that the ensemble correctly captures the magnitude of those uncertainties. That ideal situation is indicated by a small difference between and in addition to the small difference between and . The ECMWF ensemble satisfies this requirement at the forecast times where the difference between and is small. As for the relationship between and in the other ensembles, with the exception of the JMA ensemble, tends to overestimate at the analysis and the short forecast times. In other words, the ensembles compensate for part of the loss of the magnitude that results from not capturing all uncertain analysis and forecast features by overinflating the magnitude of the correctly captured features. For most ensembles, this strategy pays off at later forecast times in the form of a good match between and (NCEP, UKMO, and CMA), or between and (CMC and KMA). The fact that for the JMA ensemble remains smaller than at all forecast times suggests that the magnitude of the initial perturbations in that ensemble could be increased somewhat without negative effects on the performance of the ensemble. The CMC ensemble tends to underestimate the magnitude of the uncertainties that it captures correctly.

A comparison of the two left panels of Fig. 3 suggests that the outstanding performance of the ECMWF ensemble is due to a combination of a faster convergence of to in the first 48–72 forecast hours and a continued convergence beyond those forecast times, leading to an almost perfect fit of the two curves beyond forecast time 192 h.

b. Diagnostics based on averages over all forecasts

Figures 17 and 18 show the spatiotemporal evolution of in the forecasts for the ECMWF and the CMC ensembles, respectively. The main spatial patterns in these figures are very similar to those of in Figs. 6 and 18, but the magnitudes of the patterns are typically smaller for . This result indicates that the ensembles correctly predict the regions of main forecast uncertainty, but they do not capture all uncertain forecast features in those regions. The ratio between and tends to be smaller for the CMC than the ECMWF ensemble, which indicates that the CMC ensemble is less efficient in capturing uncertain forecast features.

Fig. 17.
Fig. 17.

Spatiotemporal evolution of (color shades) in the ECMWF ensemble forecasts. Also shown is the time mean of the geopotential analyses at 500 hPa (contours). Dashes indicate the southern boundary of the region used in the computation of the spatial averages shown in Figs. 3 and 4.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

Fig. 18.
Fig. 18.

As in Fig. 17, but for the CMC ensemble.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

A particularly good example for the ensembles capturing a large part of the structure of the most important local uncertainty is the matching pair of local maxima in and over Iceland for the ECMWF ensemble at forecast time 360 h. This shows that the shift of the related maximum in the spread (Fig. 8) is the result of not capturing all uncertain forecast features over Iceland and overestimating the spread over Newfoundland. Another example is the local maximum of over the northeast Pacific and the related maxima of and . In that case, the maxima of is at the right location, but its magnitude is overinflated, most likely by the representation of the effect of model uncertainty, to compensate for the loss of spread due to the ensemble not capturing all uncertain forecast features.

An overinflation of in large regions is a general property of the ECMWF ensemble (Fig. 19). This result suggests that while the parameters of the algorithms for the representation of the effect of model uncertainty can be efficiently tuned to achieve the good match shown between , , and (Fig. 3), they do not guarantee a near-optimal representation of the local structure of the forecast uncertainty. We note that the CMC ensemble does not suffer from a similar problem.

Fig. 19.
Fig. 19.

The spatiotemporal evolution of the local ratio between and (color shades) in the ECMWF ensemble forecasts. Also shown is the time mean of the geopotential analyses at 500 hPa (contours). Dashes indicate the southern boundary of the region used in the computation of the spatial averages shown in Figs. 3 and 4.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-15-0293.1

8. Conclusions

We proposed a description of the forecast uncertainty growth process summarized by an itemized list in section 6c, which emphasizes the earlier rapid growth of uncertainties at the synoptic scales and the later shift of the dominant error growth toward the large scales in the spatiotemporal evolution of forecast uncertainty. We found that the TIGGE ensembles were able to capture the main characteristics of the error growth process. The results also showed, however, that the accuracy of the quantitative prediction of the forecast uncertainty was strongly system dependent. While the best performing ensembles did well with respect to the spatiotemporally averaged diagnostics, the location-dependent temporally averaged diagnostics revealed that even the best performing ensembles had large errors in the representation of the local properties of the uncertainty.

A result that was particularly interesting from both a theoretical and a practical point of view was the typical growth of the error in the prediction of the mean state with increasing forecast time. Such a drift of the predicted probability distribution of the state in state space greatly reduces the utility of the longer range (week 2 and beyond) ensemble predictions. If the growth of the mean forecast uncertainty is primarily due to shortcomings of the models and/or the ensemble generation techniques, there is hope that it can be greatly reduced by refining the models and the ensemble generation techniques. However, if it is due to some fundamental properties of the atmospheric dynamics, it may turn out to be a major barrier to the extension of numerical weather forecasts into the subseasonal-to-seasonal forecast range.

Our analysis showed that the growing errors in the prediction of the mean state were dominantly due to errors in the prediction of low-frequency changes in the large-scale flow. One school of thoughts suggests that low-frequency variability is a manifestation of the internal (chaotic) variability of the atmospheric dynamics at the large scales (e.g., Legras and Ghil 1985). If this was true and the models fully captured the internal variability of the atmosphere, the systematic errors in the prediction of the mean state would not increase with forecast time and the ensembles were able to capture the related forecast uncertainty. Hence, one potential explanation for our result is that the models cannot fully capture the internal variability of the atmosphere; for instance, due to poor representation of the atmospheric dynamics in the tropics and/or atmosphere–ocean interactions. The results of Reynolds et al. (2015), for example, suggest that ensemble forecasts tend to lose temporal variability with increasing forecast time. Another potential explanation is that the low-frequency variability is not dominated by internal variability of the atmospheric dynamics. In particular, some authors (e.g., Sura et al. 2005) have argued that low- frequency variability may be due to state-dependent variations of stochastic feedbacks. Such state-dependent stochastic feedback in a long-range prediction is provided by the high-frequency (synoptic scale) transients.

Acknowledgments

This study was supported by the National Science Foundation (Grant ATM-AGS-1237613), while the TIGGE data were downloaded from the ECMWF data portal.

REFERENCES

  • Berner, J., G. J. Shutts, M. Leutbecher, and T. N. Palmer, 2009: A spectral stochastic kinetic energy backscatter scheme and its impact on flow-dependent predictability in the ECMWF ensemble prediction system. J. Atmos. Sci., 66, 603626, doi:10.1175/2008JAS2677.1.

    • Search Google Scholar
    • Export Citation
  • Berner, J., S. Y. Ha, J. P. Hacker, A. Fournier, and C. Snyder, 2011: Model uncertainty in a mesoscale ensemble prediction system: Stochastic versus multiphysics representations. Mon. Wea. Rev., 139, 19721995, doi:10.1175/2010MWR3595.1.

    • Search Google Scholar
    • Export Citation
  • Bishop, C., B. Etherton, and S. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129, 420436, doi:10.1175/1520-0493(2001)129<0420:ASWTET>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Bougeault, P., and Coauthors, 2010: The THORPEX Interactive Grand Global Ensemble (TIGGE). Bull. Amer. Meteor. Soc., 91, 10591072, doi:10.1175/2010BAMS2853.1.

    • Search Google Scholar
    • Export Citation
  • Bowler, N. E., and K. Mylne, 2009: Ensemble transform Kalman filter perturbations for a regional ensemble prediction system. Quart. J. Roy. Meteor. Soc., 135, 757766, doi:10.1002/qj.404.

    • Search Google Scholar
    • Export Citation
  • Bowler, N. E., A. Arribas, S. E. Beare, K. R. Mylne, and G. J. Shutts, 2009: The local ETKF and SKEB: Upgrades to the MOGREPS short-range ensemble prediction system. Quart. J. Roy. Meteor. Soc., 135, 767776, doi:10.1002/qj.394.

    • Search Google Scholar
    • Export Citation
  • Buizza, R., J. Tribbia, F. Molteni, and T. Palmer, 1993: Computation of optimal unstable structures for a numerical weather prediction model. Tellus, 45A, 388407, doi:10.1034/j.1600-0870.1993.t01-4-00005.x.

    • Search Google Scholar
    • Export Citation
  • Buizza, R., M. Milleer, and T. N. Palmer, 1999: Stochastic representation of model uncertainties in the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 125, 28872908, doi:10.1002/qj.49712556006.

    • Search Google Scholar
    • Export Citation
  • Buizza, R., M. Leutbecher, and L. Isaksen, 2008: Potential use of an ensemble of analyses in the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 134, 20512066, doi:10.1002/qj.346.

    • Search Google Scholar
    • Export Citation
  • Chang, E., 2000: Wave packets and life cycles of troughs in the upper troposphere. Part II: Seasonal and hemispheric variations. Mon. Wea. Rev., 128, 2550, doi:10.1175/1520-0493(2000)128<0025:WPALCO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Charron, M., G. Pellerin, L. Spacek, P. L. Houtekamer, N. Gagnon, H. L. Mitchell, and L. Michelin, 2010: Toward random sampling of model error in the Canadian ensemble prediction system. Mon. Wea. Rev., 138, 18771901, doi:10.1175/2009MWR3187.1.

    • Search Google Scholar
    • Export Citation
  • Dalcher, A., and E. Kalnay, 1987: Error growth and predictability in operational ECMWF forecasts. Tellus, 39A, 474491, doi:10.1111/j.1600-0870.1987.tb00322.x.

    • Search Google Scholar
    • Export Citation
  • Durran, D., and M. Gingrich, 2014: Atmospheric predictability: Why butterflies are not of practical importance. J. Atmos. Sci., 71, 24762488, doi:10.1175/JAS-D-14-0007.1.

    • Search Google Scholar
    • Export Citation
  • Epstein, E., 1969: Stochastic dynamic prediction. Tellus, 21A, 739759, doi:10.1111/j.2153-3490.1969.tb00483.x.

  • Hou, D., Z. Toth, Y. Zhu, and W. Yang, 2008: Impact of a stochastic perturbation scheme on NCEP global ensemble forecast system. 19th Conf. on Probability and Statistics, New Orleans, LA, Amer. Meteor. Soc., 1.1. [Available online at https://ams.confex.com/ams/pdfpapers/134165.pd.]

  • Houtekamer, P. L., 2002: The use of multiple parameterizations in ensembles. Proc. ECMWF Workshop on Model Uncertainty, Reading, United Kingdom, ECMWF, 163174.

  • Hunt, B., E. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230, 112126, doi:10.1016/j.physd.2006.11.008.

    • Search Google Scholar
    • Export Citation
  • Kuhl, D., and Coauthors, 2007: Assessing predictability with a local ensemble Kalman filter. J. Atmos. Sci., 64, 11161140, doi:10.1175/JAS3885.1.

    • Search Google Scholar
    • Export Citation
  • Legras, B., and M. Ghil, 1985: Persistent anomalies, blocking and variations in atmospheric predictability. J. Atmos. Sci., 42, 433471, doi:10.1175/1520-0469(1985)042<0433:PABAVI>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Leith, C., 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev., 102, 409418, doi:10.1175/1520-0493(1974)102<0409:TSOMCF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Leutbecher, M., and S. T. K. Lang, 2014: On the reliability of ensemble variance in subspaces defined by singular vectors. Quart. J. Roy. Meteor. Soc., 140, 14531466, doi:10.1002/qj.2229.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1969a: Atmospheric predictability as revealed by naturally occurring analogues. J. Atmos. Sci., 26, 636646, doi:10.1175/1520-0469(1969)26<636:APARBN>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1969b: The predictability of a flow which possesses many scales of motion. Tellus, 21A, 289307, doi:10.1111/j.2153-3490.1969.tb00444.x.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1982: Atmospheric predictability experiments with a large numerical model. Tellus, 34A, 505513, doi:10.1111/j.2153-3490.1982.tb01839.x.

    • Search Google Scholar
    • Export Citation
  • Magnusson, L., and E. Kallen, 2013: Factors influencing the skill improvements in the ECMWF forecasting system. Mon. Wea. Rev., 141, 31423153, doi:10.1175/MWR-D-12-00318.1.

    • Search Google Scholar
    • Export Citation
  • Molteni, F., and T. Palmer, 1993: Predictability and finite-time instability of the northern winter circulation. Quart. J. Roy. Meteor. Soc., 119, 269298, doi:10.1002/qj.49711951004.

    • Search Google Scholar
    • Export Citation
  • Mureau, R., F. Molteni, and T. Palmer, 1993: Ensemble prediction using dynamically conditioned perturbations. Quart. J. Roy. Meteor. Soc., 119, 299323, doi:10.1002/qj.49711951005.

    • Search Google Scholar
    • Export Citation
  • Nicolis, C., P. Perdigao, and S. Vannitsem, 2009: Dynamics of prediction errors under the combined effect of initial condition and model errors. J. Atmos. Sci., 66, 766778, doi:10.1175/2008JAS2781.1.

    • Search Google Scholar
    • Export Citation
  • Orlanski, I., and J. Katzfey, 1991: The life cycle of a cyclone wave in the Southern Hemisphere. Part I: Eddy energy budget. J. Atmos. Sci., 48, 19721998, doi:10.1175/1520-0469(1991)048<1972:TLCOAC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Orlanski, I., and E. Chang, 1993: Ageostrophic geopotential fluxes and upstream development of baroclinic waves. J. Atmos. Sci., 50, 212225, doi:10.1175/1520-0469(1993)050<0212:AGFIDA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Orrell, D., L. Smith, J. Barkmeijer, and T. Palmer, 2001: Model error in weather forecasting. Nonlinear Processes Geophys., 8, 357371, doi:10.5194/npg-8-357-2001.

    • Search Google Scholar
    • Export Citation
  • Ott, E., and Coauthors, 2004: A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A, 415428, doi:10.1111/j.1600-0870.2004.00076.x.

    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., R. Buizza, F. Doblas-Reyes, T. Jung, M. Leutbecher, G. J. Shutts, M. Steinheimer, and A. Weisheimer, 2009: Stochastic parameterization and model uncertainty. ECMWF Tech. Memo. 598, European Centre for Medium-Range Weather Forecasts, 42 pp.

  • Reynolds, C., E. A. Satterfield, and C. H. Bishop, 2015: Using forecast temporal variability to evaluate model behavior. Mon. Wea. Rev., 143, 47854804, doi:10.1175/MWR-D-15-0083.1.

    • Search Google Scholar
    • Export Citation
  • Rotunno, R., and C. Snyder, 2008: A generalization of Lorenz’s model for the predictability of flows with many scales of motion. J. Atmos. Sci., 65, 10631076, doi:10.1175/2007JAS2449.1.

    • Search Google Scholar
    • Export Citation
  • Satterfield, E., and I. Szunyogh, 2010: Predictability of the performance of an ensemble forecast system: Predictability of the space of uncertainties. Mon. Wea. Rev., 138, 962981, doi:10.1175/2009MWR3049.1.

    • Search Google Scholar
    • Export Citation
  • Satterfield, E., and I. Szunyogh, 2011: Assessing the performance of an ensemble forecast system in predicting the magnitude and the spectrum of analysis and forecast uncertainties. Mon. Wea. Rev., 139, 12071223, doi:10.1175/2010MWR3439.1.

    • Search Google Scholar
    • Export Citation
  • Shutts, G. J., 2005: A kinetic energy backscatter algorithm for use in ensemble prediction systems. Quart. J. Roy. Meteor. Soc., 131, 30793102, doi:10.1256/qj.04.106.

    • Search Google Scholar
    • Export Citation
  • Shutts, G. J., 2013: Coarse graining the vorticity equation in the ECMWF integrated forecasting system: The search for kinetic energy backscatter. J. Atmos. Sci., 70, 12331241, doi:10.1175/JAS-D-12-0216.1.

    • Search Google Scholar
    • Export Citation
  • Sura, P., M. Newman, C. Penland, and P. Sardeshmukh, 2005: Multiplicative noise and non-Gaussianity: A paradigm for atmospheric regimes? J. Atmos. Sci., 62, 13911409, doi:10.1175/JAS3408.1.

    • Search Google Scholar
    • Export Citation
  • Swinbank, R., and Coauthors, 2016: The TIGGE project and its achievements. Bull. Amer. Meteor. Soc., 97, 4967, doi:10.1175/BAMS-D-13-00191.1.

    • Search Google Scholar
    • Export Citation
  • Szunyogh, I., 2014: Applicable Atmospheric Dynamics: Techniques for the Exploration of Atmospheric Dynamics. 1st ed. World Scientific, 608 pp.

  • Szunyogh, I., E. Kostelich, G. Gyarmati, D. Patil, B. Hunt, E. Kalnay, E. Ott, and J. Yorke, 2005: Assessing a local ensemble Kalman filter: Perfect model experiments with the National Centers for Environmental Prediction global model. Tellus, 57A, 528545, doi:10.1111/j.1600-0870.2005.00136.x.

    • Search Google Scholar
    • Export Citation
  • Szunyogh, I., E. Kostelich, G. Gyarmati, E. Kalnay, B. Hunt, E. Ott, E. Satterfield, and J. Yorke, 2008: A local ensemble transform Kalman filter data assimilation system for the NCEP global model. Tellus, 60A, 113130, doi:10.1111/j.1600-0870.2007.00274.x.

    • Search Google Scholar
    • Export Citation
  • Talagrand, O., 1981: A study of the dynamics of four-dimensional data assimilation. Tellus, 33A, 4360, doi:10.1111/j.2153-3490.1981.tb01729.x.

    • Search Google Scholar
    • Export Citation
  • Tennant, W. J., G. J. Shutts, A. Arribas, and S. A. Thompson, 2011: Using a stochastic kinetic energy backscatter scheme to improve MOGREPS probabilistic forecast skill. Mon. Wea. Rev., 139, 11901206, doi:10.1175/2010MWR3430.1.

    • Search Google Scholar
    • Export Citation
  • Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74, 23172330, doi:10.1175/1520-0477(1993)074<2317:EFANTG>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Toth, Z., and E. Kalnay, 1997: Ensemble forecasting at NCEP and the breeding method. Mon. Wea. Rev., 125, 32973319, doi:10.1175/1520-0493(1997)125<3297:EFANAT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Tribbia, J. J., and D. P. Baumhefner, 2004: Scale interactions and atmospheric predictability: An updated perspective. Mon. Wea. Rev., 132, 703713, doi:10.1175/1520-0493(2004)132<0703:SIAAPA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Vannitsem, S., and Z. Toth, 2002: Short-term dynamics of model errors. J. Atmos. Sci., 59, 25942604, doi:10.1175/1520-0469(2002)059<2594:STDOME>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Wang, X., and C. Bishop, 2003: A comparison of breeding and ensemble transform Kalman filter ensemble forecast schemes. J. Atmos. Sci., 60, 11401158, doi:10.1175/1520-0469(2003)060<1140:ACOBAE>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Wei, M., Z. Toth, R. Wobus, and Y. Zhu, 2008: Initial perturbations based on the ensemble transform (ET) technique in the NCEP global operational forecast system. Tellus, 60A, 6279, doi:10.1111/j.1600-0870.2007.00273.x.

    • Search Google Scholar
    • Export Citation
  • Zagar, N., R. Buizza, and J. Tribbia, 2015: A three-dimensional multivariate modal analysis of atmospheric predictability with application to the ECMWF ensemble. J. Atmos. Sci., 72, 44234444, doi:10.1175/JAS-D-15-0061.1.

    • Search Google Scholar
    • Export Citation
1

For instance, the underestimation of by at analysis time for the ensemble is even smaller when the ECMWF analyses are used as proxy for the true states.

Save