Editorial Type:
Article
Article Type:
Research Article

A Spatial View of Ensemble Spread in Convection Permitting Ensembles

Seonaid R. A. Dey Department of Meteorology, University of Reading, Reading, United Kingdom

Search for other papers by Seonaid R. A. Dey in
Current site
Google Scholar
PubMed Close
,
Giovanni Leoncini Met Office, Reading, United Kingdom

Search for other papers by Giovanni Leoncini in
Current site
Google Scholar
PubMed Close
,
Nigel M. Roberts Met Office, Reading, United Kingdom

Search for other papers by Nigel M. Roberts in
Current site
Google Scholar
PubMed Close
,
Robert S. Plant Department of Meteorology, University of Reading, Reading, United Kingdom

Search for other papers by Robert S. Plant in
Current site
Google Scholar
PubMed Close
, and
Stefano Migliorini Department of Meteorology, University of Reading, Reading, United Kingdom

Search for other papers by Stefano Migliorini in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Nov 2014
DOI:
https://doi.org/10.1175/MWR-D-14-00172.1
Page(s):
4091–4107
Restricted access

Abstract

With movement toward kilometer-scale ensembles, new techniques are needed for their characterization. A new methodology is presented for detailed spatial ensemble characterization using the fractions skill score (FSS). To evaluate spatial forecast differences, the average and standard deviation are taken of the FSS calculated over all ensemble member–member pairs at different scales and lead times. These methods were found to give important information about the ensemble behavior allowing the identification of useful spatial scales, spinup times for the model, and upscale growth of errors and forecast differences. The ensemble spread was found to be highly dependent on the spatial scales considered and the threshold applied to the field. High thresholds picked out localized and intense values that gave large temporal variability in ensemble spread: local processes and undersampling dominate for these thresholds. For lower thresholds the ensemble spread increases with time as differences between the ensemble members upscale. Two convective cases were investigated based on the Met Office United Model run at 2.2-km resolution. Different ensemble types were considered: ensembles produced using the Met Office Global and Regional Ensemble Prediction System (MOGREPS) and an ensemble produced using different model physics configurations. Comparison of the MOGREPS and multiphysics ensembles demonstrated the utility of spatial ensemble evaluation techniques for assessing the impact of different perturbation strategies and the need for assessing spread at different, believable, spatial scales.

Current affiliation: Aspen Re, Zürich, Switzerland.

Corresponding author address: Seonaid Dey, Department of Meteorology, University of Reading, Earley Gate, P.O. Box 243, Reading RG6 6BB, United Kingdom. E-mail: s.dey@pgr.reading.ac.uk

Abstract

With movement toward kilometer-scale ensembles, new techniques are needed for their characterization. A new methodology is presented for detailed spatial ensemble characterization using the fractions skill score (FSS). To evaluate spatial forecast differences, the average and standard deviation are taken of the FSS calculated over all ensemble member–member pairs at different scales and lead times. These methods were found to give important information about the ensemble behavior allowing the identification of useful spatial scales, spinup times for the model, and upscale growth of errors and forecast differences. The ensemble spread was found to be highly dependent on the spatial scales considered and the threshold applied to the field. High thresholds picked out localized and intense values that gave large temporal variability in ensemble spread: local processes and undersampling dominate for these thresholds. For lower thresholds the ensemble spread increases with time as differences between the ensemble members upscale. Two convective cases were investigated based on the Met Office United Model run at 2.2-km resolution. Different ensemble types were considered: ensembles produced using the Met Office Global and Regional Ensemble Prediction System (MOGREPS) and an ensemble produced using different model physics configurations. Comparison of the MOGREPS and multiphysics ensembles demonstrated the utility of spatial ensemble evaluation techniques for assessing the impact of different perturbation strategies and the need for assessing spread at different, believable, spatial scales.

Current affiliation: Aspen Re, Zürich, Switzerland.

Corresponding author address: Seonaid Dey, Department of Meteorology, University of Reading, Earley Gate, P.O. Box 243, Reading RG6 6BB, United Kingdom. E-mail: s.dey@pgr.reading.ac.uk
Save
Cover Monthly Weather Review
Monthly Weather Review

  • Ancell, B. C., 2013: Nonlinear characteristics of ensemble perturbation evolution and their application to forecasting high-impact events. Wea. Forecasting, 28, 1353–1365, doi:10.1175/WAF-D-12-00090.1.

    • Search Google Scholar
    • Export Citation

  • Baker, L., A. Rudd, S. Migliorini, and R. Bannister, 2014: Representation of model error in a convective-scale ensemble prediction system. Nonlinear Processes Geophys., 21, 19–39, doi:10.5194/npg-21-19-2014.

    • Search Google Scholar
    • Export Citation

  • Berner, J., S.-Y. Ha, J. Hacker, A. Fournier, and C. Snyder, 2011: Model uncertainty in a mesoscale ensemble prediction system: Stochastic versus multiphysics representations. Mon. Wea. Rev., 139, 1972–1995, doi:10.1175/2010MWR3595.1.

    • Search Google Scholar
    • Export Citation

  • Bowler, N. E., A. Arribas, K. R. Mylne, K. B. Robertson, and S. E. Beare, 2008: The MOGREPS short-range ensemble prediction system. Quart. J. Roy. Meteor. Soc., 134, 703–722, doi:10.1002/qj.234.

    • Search Google Scholar
    • Export Citation

  • Bowler, N. E., A. Arribas, S. E. Beare, K. R. Mylne, and G. J. Shutts, 2009: The local ETKF and SKEB: Upgrades to the MOGREPS short-range ensemble prediction system. Quart. J. Roy. Meteor. Soc., 135, 767–776, doi:10.1002/qj.394.

    • Search Google Scholar
    • Export Citation

  • Caron, J.-F., 2013: Mismatching perturbations at the lateral boundaries in limited-area ensemble forecasting: A case study. Mon. Wea. Rev., 141, 356–374, doi:10.1175/MWR-D-12-00051.1.

    • Search Google Scholar
    • Export Citation

  • Clark, A. J., and Coauthors, 2011: Probabilistic precipitation forecast skill as a function of ensemble size and spatial scale in a convection-allowing ensemble. Mon. Wea. Rev., 139, 1410–1418, doi:10.1175/2010MWR3624.1.

    • Search Google Scholar
    • Export Citation

  • Craig, G. C., C. Keil, and D. Leuenberger, 2012: Constraints on the impact of radar rainfall data assimilation on forecasts of cumulus convection. Quart. J. Roy. Meteor. Soc., 138, 340–352, doi:10.1002/qj.929.

    • Search Google Scholar
    • Export Citation

  • Davies, T., M. J. P. Cullen, A. J. Malcolm, M. H. Mawson, A. Staniforth, A. A. White, and N. Wood, 2005: A new dynamical core for the Met Office’s global and regional modelling of the atmosphere. Quart. J. Roy. Meteor. Soc., 131, 1759–1782, doi:10.1256/qj.04.101.

    • Search Google Scholar
    • Export Citation

  • Derbyshire, S., A. Maidens, S. Milton, R. Stratton, and M. Willett, 2011: Adaptive detrainment in a convective parametrization. Quart. J. Roy. Meteor. Soc., 137, 1856–1871, doi:10.1002/qj.875.

    • Search Google Scholar
    • Export Citation

  • Done, J. M., G. C. Craig, S. L. Gray, and P. A. Clark, 2012: Case-to-case variability of predictability of deep convection in a mesoscale model. Quart. J. Roy. Meteor. Soc., 138, 638–648, doi:10.1002/qj.943.

    • Search Google Scholar
    • Export Citation

  • Duc, L., K. Saito, and H. Seko, 2013: Spatial-temporal fractions verification for high-resolution ensemble forecasts. Tellus, 65A, 18171, doi:10.3402/tellusa.v65i0.18171.

    • Search Google Scholar
    • Export Citation

  • Duda, J. D., and W. A. Gallus Jr., 2013: The impact of large-scale forcing on skill of simulated convective initiation and upscale evolution with convection-allowing grid spacings in the WRF. Wea. Forecasting, 28, 994–1018, doi:10.1175/WAF-D-13-00005.1.

    • Search Google Scholar
    • Export Citation

  • Ebert, E. E., 2008: Fuzzy verification of high-resolution gridded forecasts: A review and proposed framework. Meteor. Appl., 15, 51–64, doi:10.1002/met.25.

    • Search Google Scholar
    • Export Citation

  • Edwards, J. M., and A. Slingo, 1996: Studies with a flexible new radiation code. I: Choosing a configuration for a large-scale model. Quart. J. Roy. Meteor. Soc., 122, 689–719, doi:10.1002/qj.49712253107.

    • Search Google Scholar
    • Export Citation

  • Ehrendorfer, M., 1997: Predicting the uncertainty of numerical weather forecasts: A review. Meteor. Z., 6, 147–183.

  • Essery, R., M. Best, and P. Cox, 2001: MOSES 2.2 technical documentation. Tech. Rep., Hadley Centre Tech. Note 30, 44 pp.

  • Gebhardt, C., S. Theis, M. Paulat, and Z. Ben Bouallègue, 2011: Uncertainties in COSMO-DE precipitation forecasts introduced by model perturbations and variation of lateral boundaries. Atmos. Res., 100, 168–177, doi:10.1016/j.atmosres.2010.12.008.

    • Search Google Scholar
    • Export Citation

  • Gilleland, E., D. Ahijevych, B. G. Brown, B. Casati, and E. E. Ebert, 2009: Intercomparison of spatial forecast verification methods. Wea. Forecasting, 24, 1416–1430, doi:10.1175/2009WAF2222269.1.

    • Search Google Scholar
    • Export Citation

  • Golding, B. W., 1998: Nimrod: A system for generating automated very short range forecasts. Meteor. Appl., 5, 1–16, doi:10.1017/S1350482798000577.

    • Search Google Scholar
    • Export Citation

  • Gregory, D., and P. R. Rowntree, 1990: A mass flux convection scheme with representation of cloud ensemble characteristics and stability-dependent closure. Mon. Wea. Rev., 118, 1483–1506, doi:10.1175/1520-0493(1990)118<1483:AMFCSW>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Hacker, J. P., C. Snyder, S.-Y. Ha, and M. Pocernich, 2011: Linear and non-linear response to parameter variations in a mesoscale model. Tellus, 63A, 429–444, doi:10.1111/j.1600-0870.2010.00505.x.

    • Search Google Scholar
    • Export Citation

  • Hanley, K. E., D. J. Kirshbaum, S. E. Belcher, N. M. Roberts, and G. Leoncini, 2011: Ensemble predictability of an isolated mountain thunderstorm in a high-resolution model. Quart. J. Roy. Meteor. Soc., 137, 2124–2137, doi:10.1002/qj.877.

    • Search Google Scholar
    • Export Citation

  • Hanley, K. E., D. J. Kirshbaum, N. Roberts, and G. Leoncini, 2013: Sensitivities of a squall line over central Europe in a convective-scale ensemble. Mon. Wea. Rev., 141, 112–133, doi:10.1175/MWR-D-12-00013.1.

    • Search Google Scholar
    • Export Citation

  • Harrison, D. L., S. J. Driscoll, and M. Kitchen, 2000: Improving precipitation estimates from weather radar using quality control and correction techniques. Meteor. Appl., 7, 135–144, doi:10.1017/S1350482700001468.

    • Search Google Scholar
    • Export Citation

  • Hohenegger, C., and C. Schär, 2007a: Atmospheric predictability at synoptic versus cloud-resolving scales. Bull. Amer. Meteor. Soc., 88, 1783–1793, doi:10.1175/BAMS-88-11-1783.

    • Search Google Scholar
    • Export Citation

  • Hohenegger, C., and C. Schär, 2007b: Predictability and error growth dynamics in cloud-resolving models. J. Atmos. Sci., 64, 4467–4478, doi:10.1175/2007JAS2143.1.

    • Search Google Scholar
    • Export Citation

  • Hohenegger, C., D. Lüthi, and C. Schär, 2006: Predictability mysteries in cloud-resolving models. Mon. Wea. Rev., 134, 2095–2107, doi:10.1175/MWR3176.1.

    • Search Google Scholar
    • Export Citation

  • Johnson, A., and X. Wang, 2013: Object-based evaluation of a storm-scale ensemble during the 2009 NOAA hazardous weather testbed spring experiment. Mon. Wea. Rev., 141, 1079–1098, doi:10.1175/MWR-D-12-00140.1.

    • Search Google Scholar
    • Export Citation

  • Johnson, A., and Coauthors, 2014: Multiscale characteristics and evolution of perturbations for warm season convection-allowing precipitation forecasts: Dependence on background flow and method of perturbation. Mon. Wea. Rev., 142, 1053–1073, doi:10.1175/MWR-D-13-00204.1.

    • Search Google Scholar
    • Export Citation

  • Keil, C., and G. C. Craig, 2011: Regime-dependent forecast uncertainty of convective precipitation. Meteor. Z., 20, 145–151, doi:10.1127/0941-2948/2011/0219.

    • Search Google Scholar
    • Export Citation

  • Keil, C., F. Heinlein, and G. C. Craig, 2014: The convective adjustment time-scale as indicator of predictability of convective precipitation. Quart. J. Roy. Meteor. Soc., 140, 480–490, doi:10.1002/qj.2143.

    • Search Google Scholar
    • Export Citation

  • Kühnlein, C., C. Keil, G. C. Craig, and C. Gebhardt, 2014: The impact of downscaled initial condition perturbations on convective-scale ensemble forecasts of precipitation. Quart. J. Roy. Meteor. Soc., 140, 1552–1562, doi:10.1002/qj.2238.

    • Search Google Scholar
    • Export Citation

  • Lean, H. W., P. A. Clark, M. Dixon, N. M. Roberts, A. Fitch, R. Forbes, and C. Halliwell, 2008: Characteristics of high-resolution versions of the Met Office Unified Model for forecasting convection over the United Kingdom. Mon. Wea. Rev., 136, 3408–3424, doi:10.1175/2008MWR2332.1.

    • Search Google Scholar
    • Export Citation

  • Leoncini, G., R. S. Plant, S. L. Gray, and P. A. Clark, 2010: Perturbation growth at the convective scale for CSIP IOP18. Quart. J. Roy. Meteor. Soc., 136, 653–670, doi:10.1002/qj.587.

    • Search Google Scholar
    • Export Citation

  • Leoncini, G., N. Roberts, and B. Golding, 2011: 8th July 2011 floods in Scotland. Scottish Environment Protection Agency Rep., Met Office, 18 pp.

  • Leoncini, G., R. S. Plant, S. L. Gray, and P. A. Clark, 2013: Ensemble forecasts of a flood-producing storm: Comparison of the influence of model-state perturbations and parameter modifications. Quart. J. Roy. Meteor. Soc., 139, 198–211, doi:10.1002/qj.1951.

    • Search Google Scholar
    • Export Citation

  • Lock, A., A. Brown, M. Bush, G. Martin, and R. Smith, 2000: A new boundary layer mixing scheme. Part I: Scheme description and single-column model tests. Mon. Wea. Rev., 128, 3187–3199, doi:10.1175/1520-0493(2000)128<3187:ANBLMS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Lorenz, E. N., 1969: The predictability of a flow which possesses many scales of motion. Tellus, 21A, 289–307, doi:10.1111/j.2153-3490.1969.tb00444.x.

    • Search Google Scholar
    • Export Citation

  • Martin, W. J., and M. Xue, 2006: Sensitivity analysis of convection of the 24 May 2002 IHOP case using very large ensembles. Mon. Wea. Rev., 134, 192–207, doi:10.1175/MWR3061.1.

    • Search Google Scholar
    • Export Citation

  • Mass, C. F., D. Ovens, K. Westrick, and B. A. Colle, 2002: Does increasing horizontal resolution produce more skillful forecasts? Bull. Amer. Meteor. Soc., 83, 407–430, doi:10.1175/1520-0477(2002)083<0407:DIHRPM>2.3.CO;2.

    • Search Google Scholar
    • Export Citation

  • Migliorini, S., M. Dixon, R. Bannister, and S. Ballard, 2011: Ensemble prediction for nowcasting with a convection-permitting model-I: Description of the system and the impact of radar-derived surface precipitation rates. Tellus, 63A, 468–496, doi:10.1111/j.1600-0870.2010.00503.x.

    • Search Google Scholar
    • Export Citation

  • Mittermaier, M., and N. Roberts, 2010: Intercomparison of spatial forecast verification methods: Identifying skillful spatial scales using the fractions skill score. Wea. Forecasting, 25, 343–354, doi:10.1175/2009WAF2222260.1.

    • Search Google Scholar
    • Export Citation

  • Mylne, K., 2013: Scientific framework for the ensemble prediction system for the UKV. MOSAC Paper 18.6, Met Office, 12 pp. [Available online at http://www.metoffice.gov.uk/media/pdf/q/0/MOSAC_18.6_Mylne.pdf.]

  • Palmer, T. N., 2000: Predicting uncertainty in forecasts of weather and climate. Rep. Prog. Phys., 63, 71–116, doi:10.1088/0034-4885/63/2/201.

    • Search Google Scholar
    • Export Citation

  • Rezacova, D., P. Zacharov, and Z. Sokol, 2009: Uncertainty in the area-related QPF for heavy convective precipitation. Atmos. Res., 93, 238–246, doi:10.1016/j.atmosres.2008.12.005.

    • Search Google Scholar
    • Export Citation

  • Roberts, N. M., 2003: The impact of a change to the use of the convection scheme to high resolution simulations of convective events (stage 2 report from the storm scale numerical modelling project). Forecasting Research Tech. Rep. 407, Joint Centre for Mesoscale Meteorology Rep. 142, Met Office, 30 pp. [Available online at http://www.metoffice.gov.uk/media/pdf/0/8/FRTR407.pdf.]

  • Roberts, N. M., 2008: Assessing the spatial and temporal variation in the skill of precipitation forecasts from an NWP model. Meteor. Appl., 15, 163–169, doi:10.1002/met.57.

    • Search Google Scholar
    • Export Citation

  • Roberts, N. M., and H. W. Lean, 2008: Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Mon. Wea. Rev., 136, 78–97, doi:10.1175/2007MWR2123.1.

    • Search Google Scholar
    • Export Citation

  • Schwartz, C. S., and Coauthors, 2010: Toward improved convection-allowing ensembles: Model physics sensitivities and optimizing probabilistic guidance with small ensemble membership. Wea. Forecasting, 25, 263–280, doi:10.1175/2009WAF2222267.1.

    • Search Google Scholar
    • Export Citation

  • Stensrud, D. J., J.-W. Bao, and T. T. Warner, 2000: Using initial condition and model physics perturbations in short-range ensemble simulations of mesoscale convective systems. Mon. Wea. Rev., 128, 2077–2107, doi:10.1175/1520-0493(2000)128<2077:UICAMP>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Surcel, M., I. Zawadzki, and M. K. Yau, 2014: On the filtering properties of ensemble averaging for storm-scale precipitation forecasts. Mon. Wea. Rev., 142, 1093–1105, doi:10.1175/MWR-D-13-00134.1.

    • Search Google Scholar
    • Export Citation

  • Vié, B., G. Molinié, O. Nuissier, B. Vincendon, V. Ducrocq, F. Bouttier, and E. Richard, 2012: Hydro-meteorological evaluation of a convection-permitting ensemble prediction system for Mediterranean heavy precipitating events. Nat. Hazards Earth Syst., 12, 2631–2645, doi:10.5194/nhess-12-2631-2012.

    • Search Google Scholar
    • Export Citation

  • Walser, A. and C. Schär, 2004: Convection-resolving precipitation forecasting and its predictability in alpine river catchments. J. Hydrol., 288, 57–73.

  • Walser, A., D. Lüthi, and C. Schär, 2004: Predictability of precipitation in a cloud-resolving model. Mon. Wea. Rev., 132, 560–577, doi:10.1175/1520-0493(2004)132<0560:POPIAC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Wilkinson, J., 2011: The large-scale precipitation parametrization scheme. Unified Model Documentation Paper 26, Unified Model version: 8.0, Met Office, 53 pp.

  • Wilson, D. R., and S. P. Ballard, 1999: A microphysically based precipitation scheme for the UK Meteorological Office Unified Model. Quart. J. Roy. Meteor. Soc., 125, 1607–1636, doi:10.1002/qj.49712555707.

    • Search Google Scholar
    • Export Citation

  • Zacharov, P., and D. Rezacova, 2009: Using the fractions skill score to assess the relationship between an ensemble QPF spread and skill. Atmos. Res., 94, 684–693, doi:10.1016/j.atmosres.2009.03.004.

    • Search Google Scholar
    • Export Citation

  • Zhang, F., 2005: Dynamics and structure of mesoscale error covariance of a winter cyclone estimated through short-range ensemble forecasts. Mon. Wea. Rev., 133, 2876–2893, doi:10.1175/MWR3009.1.

    • Search Google Scholar
    • Export Citation

  • Zimmer, M., G. C. Craig, C. Keil, and H. Wernli, 2011: Classification of precipitation events with a convective response timescale and their forecasting characteristics. Geophys. Res. Lett., 38, L05802, doi:10.1029/2010GL046199.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 638 249 22
PDF Downloads 412 122 4