Multivariate Probabilistic Analysis and Predictability of Medium-Range Ensemble Weather Forecasts

Jessica Keune Meteorological Institute, University of Bonn, and Climate Monitoring Branch, Hans-Ertel Centre for Weather Research, Bonn, Germany

Search for other papers by Jessica Keune in
Current site
Google Scholar
PubMed
Close
,
Christian Ohlwein Meteorological Institute, University of Bonn, and Climate Monitoring Branch, Hans-Ertel Centre for Weather Research, Bonn, Germany

Search for other papers by Christian Ohlwein in
Current site
Google Scholar
PubMed
Close
, and
Andreas Hense Meteorological Institute, University of Bonn, Bonn, Germany

Search for other papers by Andreas Hense in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Ensemble weather forecasting has been operational for two decades now. However, the related uncertainty analysis in terms of probabilistic postprocessing still focuses on single variables, grid points, or stations. Inevitable dependencies in space and time and between variables are often ignored. To address this problem, two probabilistic postprocessing methods are presented, which are multivariate versions of Gaussian fit and kernel dressing, respectively. The multivariate case requires the estimation of a full rank, invertible covariance matrix. For this purpose, a Graphical Least Absolute Shrinkage and Selection Operators (GLASSO) estimator has been employed that is based on sparse undirected graphical models regularized by an L1 penalty term in order to parameterize the full rank inverse covariance. In all cases, the result is a multidimensional probability density. The forecasts used to test the approach are station forecasts of 2-m temperature and surface pressure from four main global ensemble prediction systems (EPS) with medium-range weather forecasts: the NCEP Global Ensemble Forecast System (GEFS), the Met Office Global and Regional Ensemble Prediction System (MOGREPS), the Canadian Meteorological Centre (CMC) Global Ensemble Prediction System (GEPS), and the ECMWF EPS. To evaluate the multivariate probabilistic postprocessing, especially the uncertainty estimates, common verification methods such as the analysis rank histogram and the continuous ranked probability score (CRPS) are applied. Furthermore, a multivariate extension of the CRPS, the energy score, allows for the verification of a complete medium-range forecast as well as for determining its predictability. It is shown that the predictability is similar for all of the examined ensemble prediction systems, whereas the GLASSO proved to be a useful tool for calibrating the commonly observed underdispersion of ensemble forecasts during the first few lead days by using information from the full covariance matrix.

Corresponding author address: Jessica Keune, Meteorological Institute, University of Bonn, Auf dem Huegel 20, 53121 Bonn, Germany. E-mail: jkeune@uni-bonn.de

Abstract

Ensemble weather forecasting has been operational for two decades now. However, the related uncertainty analysis in terms of probabilistic postprocessing still focuses on single variables, grid points, or stations. Inevitable dependencies in space and time and between variables are often ignored. To address this problem, two probabilistic postprocessing methods are presented, which are multivariate versions of Gaussian fit and kernel dressing, respectively. The multivariate case requires the estimation of a full rank, invertible covariance matrix. For this purpose, a Graphical Least Absolute Shrinkage and Selection Operators (GLASSO) estimator has been employed that is based on sparse undirected graphical models regularized by an L1 penalty term in order to parameterize the full rank inverse covariance. In all cases, the result is a multidimensional probability density. The forecasts used to test the approach are station forecasts of 2-m temperature and surface pressure from four main global ensemble prediction systems (EPS) with medium-range weather forecasts: the NCEP Global Ensemble Forecast System (GEFS), the Met Office Global and Regional Ensemble Prediction System (MOGREPS), the Canadian Meteorological Centre (CMC) Global Ensemble Prediction System (GEPS), and the ECMWF EPS. To evaluate the multivariate probabilistic postprocessing, especially the uncertainty estimates, common verification methods such as the analysis rank histogram and the continuous ranked probability score (CRPS) are applied. Furthermore, a multivariate extension of the CRPS, the energy score, allows for the verification of a complete medium-range forecast as well as for determining its predictability. It is shown that the predictability is similar for all of the examined ensemble prediction systems, whereas the GLASSO proved to be a useful tool for calibrating the commonly observed underdispersion of ensemble forecasts during the first few lead days by using information from the full covariance matrix.

Corresponding author address: Jessica Keune, Meteorological Institute, University of Bonn, Auf dem Huegel 20, 53121 Bonn, Germany. E-mail: jkeune@uni-bonn.de
Save
  • Anderson, J., 1996: A method for producing and evaluating probabilistic forecasts from ensemble model integrations. J. Climate, 9, 15181530, doi:10.1175/1520-0442(1996)009<1518:AMFPAE>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Anderson, T. W., 1958: An Introduction to Multivariate Statistical Analysis. Vol. 2. Wiley, 374 pp.

  • Baringhaus, L., and C. Franz, 2004: On a new multivariate two-sample test. J. Multivar. Anal., 88, 190206, doi:10.1016/S0047-259X(03)00079-4.

    • Search Google Scholar
    • Export Citation
  • Berkowitz, J., 2001: Testing density forecasts, with applications to risk management. J. Bus. Econ. Stat., 19, 465474, doi:10.1198/07350010152596718.

    • Search Google Scholar
    • Export Citation
  • Bougeault, P., and Coauthors, 2010: The THORPEX Interactive Grand Global Ensemble. Bull. Amer. Meteor. Soc., 91, 10591072, doi:10.1175/2010BAMS2853.1.

    • Search Google Scholar
    • Export Citation
  • Bröcker, J., 2009: Reliability, sufficiency, and the decomposition of proper scores. Quart. J. Roy. Meteor. Soc., 135, 15121519, doi:10.1002/qj.456.

    • Search Google Scholar
    • Export Citation
  • Bröcker, J., and L. A. Smith, 2008: From ensemble forecasts to predictive distribution functions. Tellus, 60A, 663678, doi:10.1111/j.1600-0870.2008.00333.x.

    • Search Google Scholar
    • Export Citation
  • Candille, G., and O. Talagrand, 2005: Evaluation of probabilistic prediction systems for a scalar variable. Quart. J. Roy. Meteor. Soc., 131, 21312150, doi:10.1256/qj.04.71.

    • Search Google Scholar
    • Export Citation
  • Chervin, R. M., 1981: On the comparison of observed and GCM simulated climate ensembles. J. Atmos. Sci., 38, 885901, doi:10.1175/1520-0469(1981)038<0885:OTCOOA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • DelSole, T., and M. K. Tippett, 2007: Predictability: Recent insights from information theory. Rev. Geophys., 45, RG4002, doi:10.1029/2006RG000202.

    • Search Google Scholar
    • Export Citation
  • Friedman, J., T. Hastie, and R. Tibshirani, 2008: Sparse inverse covariance estimation with the lasso. Biostatistics, 9, 432441, doi:10.1093/biostatistics/kxm045.

    • Search Google Scholar
    • Export Citation
  • Friedman, J., T. Hastie, and R. Tibshirani, 2011: glasso: Graphical lasso-estimation of Gaussian graphical models, version 1.7. R package. [Available online at http://CRAN.R-project.org/package=glasso.]

  • Gneiting, T., and A. E. Raftery, 2007: Strictly proper scoring rules, prediction, and estimation. J. Amer. Stat. Assoc., 102, 359378, doi:10.1198/016214506000001437.

    • Search Google Scholar
    • Export Citation
  • Gneiting, T., A. E. Raftery, A. H. Westveld III, and T. Goldman, 2005: Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Mon. Wea. Rev., 133, 10981118, doi:10.1175/MWR2904.1.

    • Search Google Scholar
    • Export Citation
  • Gneiting, T., L. I. Stanberry, E. P. Grimit, L. Held, and N. A. Johnson, 2008: Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds. Test, 17, 211235, doi:10.1007/s11749-008-0114-x.

    • Search Google Scholar
    • Export Citation
  • Grimit, E. P., T. Gneiting, V. J. Berrocal, and N. A. Johnson, 2006: The continuous ranked probability score for circular variables and its application to mesoscale forecast ensemble verification. Quart. J. Roy. Meteor. Soc., 132, 29252942, doi:10.1256/qj.05.235.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129, 550560, doi:10.1175/1520-0493(2001)129<0550:IORHFV>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., and S. J. Colucci, 1997: Verification of Eta-RSM short-range ensemble forecasts. Mon. Wea. Rev., 125, 13121327, doi:10.1175/1520-0493(1997)125<1312:VOERSR>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hasselmann, K., 1993: Optimal fingerprints for the detection of time-dependent climate change. J. Climate, 6, 1957–1971, doi:10.1175/1520-0442(1993)006<1957:OFFTDO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hersbach, H., 2000: Decomposition of the continuous ranked probability score for ensemble prediction systems. Wea. Forecasting, 15, 559570, doi:10.1175/1520-0434(2000)015<0559:DOTCRP>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Jolliffe, I. T., and D. B. Stephenson, 2011: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. 2nd ed. John Wiley & Sons, 292 pp.

  • Jonko, A. K., A. Hense, and J. J. Feddema, 2010: Effects of land cover change on the tropical circulation in a GCM. Climate Dyn., 35, 635649, doi:10.1007/s00382-009-0684-7.

    • Search Google Scholar
    • Export Citation
  • Keller, J. D., and A. Hense, 2011: A new non-Gaussian evaluation method for ensemble forecasts based on analysis rank histograms. Meteor. Z., 20, 107117, doi:10.1127/0941-2948/2011/0217.

    • Search Google Scholar
    • Export Citation
  • Mazumder, R., and T. Hastie, 2012: The graphical lasso: New insights and alternatives. Electron. J. Stat., 6, 21252149, doi:10.1214/12-EJS740.

    • Search Google Scholar
    • Export Citation
  • Meinshausen, N., and P. Bühlmann, 2006: High-dimensional graphs and variable selection with the lasso. Ann. Stat., 34, 14361462, doi:10.1214/009053606000000281.

    • Search Google Scholar
    • Export Citation
  • Menéndez, P., Y. A. I. Kourmpetis, C. J. F. ter Braak, and F. A. van Eeuwijk, 2010: Gene regulatory networks from multifactorial perturbations using Graphical Lasso: Application to the DREAM4 challenge. PLoS ONE,5, e14147, doi:10.1371/journal.pone.0014147.

  • Möller, A., A. Lenkoski, and T. L. Thorarinsdottir, 2013: Multivariate probabilistic forecasting using ensemble Bayesian model averaging and copulas. Quart. J. Roy. Meteor. Soc., 139, 982991, doi:10.1002/qj.2009.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1973: A new vector partition of the probability score. J. Appl. Meteor., 12, 595600, doi:10.1175/1520-0450(1973)012<0595:ANVPOT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1991: Forecast verification: Its complexity and dimensionality. Mon. Wea. Rev., 119, 15901601, doi:10.1175/1520-0493(1991)119<1590:FVICAD>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1993: What is a good forecast? An essay on the nature of goodness in weather forecasting. Wea. Forecasting, 8, 281293, doi:10.1175/1520-0434(1993)008<0281:WIAGFA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., and R. L. Winkler, 1987: A general framework for forecast verification. Mon. Wea. Rev., 115, 13301338, doi:10.1175/1520-0493(1987)115<1330:AGFFFV>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., and E. S. Epstein, 1989: Skill scores and correlation coefficients in model verification. Mon. Wea. Rev., 117, 572582, doi:10.1175/1520-0493(1989)117<0572:SSACCI>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Packard, N. H., J. P. Crutchfield, J. D. Farmer, and R. S. Shaw, 1980: Geometry from a time series. Phys. Rev. Lett., 45, 712716, doi:10.1103/PhysRevLett.45.712.

    • Search Google Scholar
    • Export Citation
  • Paeth, H., R. Girmes, G. Menz, and A. Hense, 2006: Improving seasonal forecasting in the low latitudes. Mon. Wea. Rev., 134, 18591879, doi:10.1175/MWR3149.1.

    • Search Google Scholar
    • Export Citation
  • R Core Team, 2013: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. [Available online at http://www.R-project.org/.]

  • Röpnack, A., A. Hense, C. Gebhardt, and D. Majewski, 2013: Bayesian model verification of NWP ensemble forecasts. Mon. Wea. Rev., 141, 375387, doi:10.1175/MWR-D-11-00350.1.

    • Search Google Scholar
    • Export Citation
  • Schölzel, C., and A. Hense, 2011: Probabilistic assessment of regional climate change in Southwest Germany by ensemble dressing. Climate Dyn., 36, 20032014, doi:10.1007/s00382-010-0815-1.

    • Search Google Scholar
    • Export Citation
  • Schuhen, N., T. L. Thorarinsdottir, and T. Gneiting, 2012: Ensemble model output statistics for wind vectors. Mon. Wea. Rev., 140, 3204–3219, doi:10.1175/MWR-D-12-00028.1.

    • Search Google Scholar
    • Export Citation
  • Silverman, B. W., 1986: Density Estimation for Statistics and Data Analysis. Chapman and Hall, 176 pp.

  • Stenseth, N. C., G. Ottersen, J. W. Hurrell, A. Mysterud, M. Lima, K. S. Chan, N. G. Yoccoz, and B. Adlandsvik, 2003: Studying climate effects on ecology through the use of climate indices: The North Atlantic Oscillation, El Niño Southern Oscillation and beyond. Proc. Roy. Soc. London,270B, 2087–2096, doi:10.1098/rspb.2003.2415.

  • Stephenson, D. B., and F. J. Doblas-Reyes, 2000: Statistical methods for interpreting Monte Carlo ensemble forecasts. Tellus, 52A, 300322, doi:10.1034/j.1600-0870.2000.d01-5.x.

    • Search Google Scholar
    • Export Citation
  • Takens, F., 1981: Detecting strange attractors in turbulence. Dynamical Systems and Turbulence, Warwick 1980, D. Rand and L.-S. Young, Eds., Lecture Notes in Mathematics, Vol. 898, Springer, 366–381.

  • Tsyplakov, A., 2013: Evaluation of probabilistic forecasts: Proper scoring rules and moments. Novosibirsk State University, Dept. of Economics Working Papers, 21 pp. [Available online at http://ssrn.com/abstract=2236605.]

  • von Storch, H., and F. W. Zwiers, 1999: Statistical Analysis in Climate Research. Cambridge University Press, 484 pp.

  • Whitaker, J. S., and A. F. Loughe, 1998: The relationship between ensemble spread and ensemble mean skill. Mon. Wea. Rev., 126, 32923302, doi:10.1175/1520-0493(1998)126<3292:TRBESA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2002: Smoothing forecast ensembles with fitted probability distributions. Quart. J. Roy. Meteor. Soc., 128, 28212836, doi:10.1256/qj.01.215.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 777 175 47
PDF Downloads 264 35 4