• Ayar, P. V., M. Vrac, and A. Mailhot, 2021: Ensemble bias correction of climate simulations: Preserving internal variability. Sci. Rep., 11, 3098, https://doi.org/10.1038/s41598-021-82715-1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Baumgart, M., P. Ghinassi, V. Wirth, T. Selz, G. C. Craig, and M. Riemer, 2019: Quantitative view on the processes governing the upscale error growth up to the planetary scale using a stochastic convection scheme. Mon. Wea. Rev., 147, 17131731, https://doi.org/10.1175/MWR-D-18-0292.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bechtold, P., N. Semane, P. Lopez, J. P. Charboureau, A. Beljaars, and N. Bormann, 2014: Representing equilibrium and nonequilibrium convection in large-scale models. J. Atmos. Sci., 71, 734753, https://doi.org/10.1175/JAS-D-13-0163.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., J. Mailhot, C. Girard, and P. Vaillancourt, 2005: Boundary layer and shallow cumulus clouds in a medium-range forecast of a large-scale weather system. Mon. Wea. Rev., 133, 19381960, https://doi.org/10.1175/MWR2958.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berner, J., S.-Y. Ha, J. P. Hacker, A. Fournier, and C. Snyder, 2011: Model uncertainty in a mesoscale ensemble prediction system: Stochastic versus multiphysics representations. Mon. Wea. Rev., 139, 19721995, https://doi.org/10.1175/2010MWR3595.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berner, J., K. R. Fossell, S.-Y. Ha, J. P. Hacker, and C. Snyder, 2015: Increasing the skill of probabilistic forecasts: Understanding performance improvements from model-error representations. Mon. Wea. Rev., 143, 12951320, https://doi.org/10.1175/MWR-D-14-00091.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berner, J., and Coauthors, 2017: Stochastic parameterization: Toward a new view of weather and climate models. Bull. Amer. Meteor. Soc., 98, 565588, https://doi.org/10.1175/BAMS-D-15-00268.1.

    • Search Google Scholar
    • Export Citation
  • Berner, J., P. D. Sardeshmukh, and H. M. Christensen, 2018: On the dynamical mechanism governing El Ninõ–Southern Oscillation irregularity. J. Climate, 31, 84018419, https://doi.org/10.1175/JCLI-D-18-0243.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bertossa, C., P. Hitchcock, A. DeGaetano, R. Plougonven, 2021: Bimodality in ensemble forecasts of 2-meter temperature: Identification. Wea. Climate Dyn., 2, 12901224, https://doi.org/10.5194/wcd-2-1209-2021.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Blackadar, A. K., 1962: The vertical distribution of wind and turbulent exchange in a neutral atmosphere. J. Geophys. Res., 67, 30953102, https://doi.org/10.1029/JZ067i008p03095.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bocquet, M., C. A. Pires, and L. Wu, 2010: Beyond Gaussian statistical modeling in geophysical data assimilation. Mon. Wea. Rev., 138, 29973023, https://doi.org/10.1175/2010MWR3164.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bougeault, P., and P. Lacarrère, 1989: Parameterization of orography-induced turbulence in a mesobeta-scale model. Mon. Wea. Rev., 117, 18721890, https://doi.org/10.1175/1520-0493(1989)117<1872:POOITI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buehner, M., 2020: Local ensemble transform Kalman filter with cross validation. Mon. Wea. Rev., 148, 22652282, https://doi.org/10.1175/MWR-D-19-0402.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buehner, M., and Coauthors, 2015: Implementation of deterministic weather forecasting systems based on ensemble–variational data assimilation at Environment Canada. Part I: The global system. Mon. Wea. Rev., 143, 25322559, https://doi.org/10.1175/MWR-D-14-00354.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buizza, R., M. Miller, and T. N. Palmer, 1999: Stochastic representation of model uncertainties in the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 125, 28872908, https://doi.org/10.1002/qj.49712556006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buizza, R., P. L. Houtekamer, G. Pierre, Z. Toth, Y. Zhu, and M. Wei, 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Mon. Wea. Rev., 133, 10761097, https://doi.org/10.1175/MWR2905.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Charnock, H., 1955: Wind stress on a water surface. Quart. J. Roy. Meteor. Soc., 81, 639640, https://doi.org/10.1002/qj.49708135027.

  • Charron, M., G. Pellerin, L. Spacek, P. L. Houtekamer, N. Gagnon, H. L. Mitchell, and L. Michelin, 2010: Toward random sampling of model error in the Canadian ensemble prediction system. Mon. Wea. Rev., 138, 18771901, https://doi.org/10.1175/2009MWR3187.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, J., J. Wang, J. Du, Y. Xia, F. Chen, and L. Hongqi, 2020: Forecast bias correction through model integration: A dynamical wholesale approach. Quart. J. Roy. Meteor. Soc., 146, 11491168, https://doi.org/10.1002/qj.3730.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Christensen, H. M., 2020: Constraining stochastic parameterization schemes using high-resolution simulations. Quart. J. Roy. Meteor. Soc., 146, 938962, https://doi.org/10.1002/qj.3717.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Christensen, H. M., I. M. Moroz, and T. N. Palmer, 2015: Stochastic and perturbed parameter representations of model uncertainty in convection parameterization. J. Atmos. Sci., 72, 25252544, https://doi.org/10.1175/JAS-D-14-0250.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Christensen, H. M., S.-J. Lock, I. M. Moroz, and T. N. Palmer, 2017: Introducing independent patterns into the stochastically perturbed parameterization tendencies (SPPT) scheme. Quart. J. Roy. Meteor. Soc., 143, 21682181, https://doi.org/10.1002/qj.3075.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cui, B., Z. Toth, Y. Zhu, and D. Hou, 2012: Bias correction for global ensemble forecast. Wea. Forecasting, 27, 396410, https://doi.org/10.1175/WAF-D-11-00011.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dai, A., and J. Wang, 1999: Diurnal and semidiurnal tides in the global surface pressure fields. J. Atmos. Sci., 56, 38743891, https://doi.org/10.1175/1520-0469(1999)056<3874:DASTIG>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deacu, D., V. Fortin, E. Klyszejko, C. Spence, and P. D. Blanken, 2012: Predicting the net basin supply to the Great Lakes with a hydrometeorological model. J. Hydrometeor., 13, 17391759, https://doi.org/10.1175/JHM-D-11-0151.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dee, D. P., and Coauthors, 2011: The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Quart. J. Roy. Meteor. Soc., 137, 553597, https://doi.org/10.1002/qj.828.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dias, J., and G. N. Kiladis, 2019: The influence of tropical forecast errors on higher latitude predictions. Geophys. Res. Lett., 46, 44504459, https://doi.org/10.1029/2019GL082812.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ferro, C. A. T., D. S. Richardson, and A. P. Weigel, 2008: On the effect of ensemble size on the discrete and continuous ranked probability scores. Meteor. Appl., 15, 1924, https://doi.org/10.1002/met.45.

    • Search Google Scholar
    • Export Citation
  • Fricker, T. E., C. A. T. Ferro, and D. B. Stephenson, 2013: Three recommendations for evaluating climate predictions. Meteor. Appl., 20, 246255, https://doi.org/10.1002/met.1409.

    • Search Google Scholar
    • Export Citation
  • Frogner, I. L., U. Andrae, P. Ollinaho, A. Hally, K. Hämäläinen, J. Kauhanen, K. L. Ivarsson, and D. Yazagi, 2022: Model uncertainty representation in a convection-permitting ensemble–SPP and SPPT in Harmon EPS. Mon. Wea. Rev., 150, 775795, https://doi.org/10.1175/MWR-D-21-0099.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gagnon, N., and Coauthors, 2013: Improvements to the Global Ensemble Prediction System (GEPS), version 2.0.3 to 3.0.0. Environment Canada Doc., 49 pp., https://collaboration.cmc.ec.gc.ca/cmc/cmoi/product_guide/docs/lib/op_systems/doc_opchanges/technote_geps300_20130213_e.pdf.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gross, M., S. Malardel, C. Jablonowski, and N. Wood, 2016: Bridging the (knowledge) gap between physics and dynamics. Bull. Amer. Meteor. Soc., 97, 137142, https://doi.org/10.1175/BAMS-D-15-00103.1.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., and S. J. Colucci, 1997: Verification of Eta-RSM short-range ensemble forecasts. Mon. Wea. Rev., 125, 13121327, https://doi.org/10.1175/1520-0493(1997)125<1312:VOERSR>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hou, D., Z. Toth, and Y. Zhu, 2006: A stochastic parameterization scheme within NCEP global ensemble forecast system. 18th Conf. on Probability and Statistics in the Atmospheric Sciences, Atlanta, GA, Amer. Meteor. Soc., 4.5, https://ams.confex.com/ams/pdfpapers/101401.pdf.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., 2011: The use of multiple parameterizations in ensembles. Proc. ECMWF Workshop on Representing Model Uncertainty and Error in Numerical Weather and Climate Prediction Models, Reading, United Kingdom, ECMWF, 163173.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., L. Lefaivre, J. Derome, H. Ritchie, and H. L. Mitchell, 1996: A system approach to ensemble prediction. Mon. Wea. Rev., 124, 12251242, https://doi.org/10.1175/1520-0493(1996)124<1225:ASSATE>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., H. L. Mitchell, and X. Deng, 2009: Model error representation in an operational ensemble Kalman filter. Mon. Wea. Rev., 137, 21262143, https://doi.org/10.1175/2008MWR2737.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jankov, I., and Coauthors, 2017: A performance comparison between multiphysics and stochastic approaches within a North American RAP ensemble. Mon. Wea. Rev., 145, 11611179, https://doi.org/10.1175/MWR-D-16-0160.1.

    • Search Google Scholar
    • Export Citation
  • Jankov, I., J. Beck, J. Wolff, M. Harrold, J. B. Olson, T. Smirnova, C. Alexander, and J. Berner, 2019: Stochastically perturbed parameterizations in an HRRR-based ensemble. Mon. Wea. Rev., 147, 153173, https://doi.org/10.1175/MWR-D-18-0092.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jeworrek, J., G. West, and R. Stull, 2021: WRF precipitation performance and predictability for systematically varied parameterizations over complex terrain. Wea. Forecasting, 36, 893913, https://doi.org/10.1175/WAF-D-20-0195.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kain, J. S., and J. M. Fritsch, 1993: Convective parameterization for mesoscale models: The Kain–Fritsch scheme. The Representation of Cumulus Convection in Numerical Models of the Atmosphere, Meteor. Monogr., No. 46, Amer. Meteor. Soc., 165170.

    • Search Google Scholar
    • Export Citation
  • Kalina, E. A., I. Jankov, T. Alcott, J. Olson, J. Beck, J. Berner, D. Dowell, and C. Alexander, 2021: A progress report on the development of the High-Resolution Rapid Refresh ensemble. Wea. Forecasting, 36, 791804, https://doi.org/10.1175/WAF-D-20-0098.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Krinner, G., V. Kharin, R. Roehrig, J. Scinocca, and F. Codron, 2020: Historically-based run-time bias corrections substantially improve model projections of 100 years of future climate change. Commun. Earth Environ., 1, 29, https://doi.org/10.1038/s43247-020-00035-0.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kuo, H. L., 1974: Further studies of the parameterization of the influence of cumulus convection on large scale flow. J. Atmos. Sci., 31, 12321240, https://doi.org/10.1175/1520-0469(1974)031<1232:FSOTPO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lang, S. T. K., S.-J. Lock, M. Leutbecher, P. Bechtold, and R. M. Forbes, 2021: Revision of the stochastically perturbed parameterizations model uncertainty scheme in the integrated forecasting system. Quart. J. Roy. Meteor. Soc., 147, 13641381, https://doi.org/10.1002/qj.3978.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lee, J. A., W. C. Kolczynski, T. C. McCandless, and S. E. Haupt, 2012: An objective methodology for configuring and down-selecting an NWP ensemble for low-level wind prediction. Mon. Wea. Rev., 140, 22702286, https://doi.org/10.1175/MWR-D-11-00065.1.

    • Search Google Scholar
    • Export Citation
  • Leutbecher, M., 2019: Ensemble size: How suboptimal is less than infinity? Quart. J. Roy. Meteor. Soc., 145, 107128, https://doi.org/10.1002/qj.3387.

  • Leutbecher, M., R. Buizza, and L. Isaksen, 2007: Ensemble forecasting and flow-dependent estimates of initial uncertainty. ECMWF Workshop on Flow-Dependent Aspects of Data Assimilation, Reading, United Kingdom, ECMWF, https://www.ecmwf.int/sites/default/files/elibrary/2007/10731-ensemble-forecasting-and-flow-dependent-estimates-initial-uncertainty.pdf.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Leutbecher, M., and Coauthors, 2017: Stochastic representations of model uncertainties at ECMWF: State of the art and future vision. Quart. J. Roy. Meteor. Soc., 143, 23152339, https://doi.org/10.1002/qj.3094.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Li, W., and Coauthors, 2019: Evaluating the MJO prediction skill from different configurations of the NCEP GEFS extended forecast. Climate Dyn., 52, 49234936, https://doi.org/10.1007/s00382-018-4423-9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lillo, S. P., and D. B. Parsons, 2017: Investigating the dynamics of error growth in ECMWF medium-range forecast busts. Quart. J. Roy. Meteor. Soc., 143, 12111226, https://doi.org/10.1002/qj.2938.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ling, J., and C. Zhang, 2013: Diabatic heating profiles in recent global reanalyses. J. Climate, 26, 33073325, https://doi.org/10.1175/JCLI-D-12-00384.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lock, S.-J., S. T. K. Lang, M. Leutbecher, R. J. Hogan, and F. Vitart, 2019: Treatment of model uncertainty from radiation by the Stochastically Perturbed Parameterization Tendencies (SPPT) scheme and associated revisions in the ECMWF ensembles. Quart. J. Roy. Meteor. Soc., 145, 7589, https://doi.org/10.1002/qj.3570.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lott, F., and M. J. Miller, 1997: A new subgrid-scale orographic drag parameterization: Its formulation and testing. Quart. J. Roy. Meteor. Soc., 123, 101127, https://doi.org/10.1002/qj.49712353704.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Madden, R. A., and P. R. Julian, 1971: Detection of a 40–50 day oscillation in the zonal wind in the tropical Pacific. J. Atmos. Sci., 28, 702708, https://doi.org/10.1175/1520-0469(1971)028<0702:DOADOI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McTaggart-Cowan, R., and Coauthors, 2019: Modernization of atmospheric physics in Canadian NWP. J. Adv. Model. Earth Syst., 11, 35933635, https://doi.org/10.1029/2019MS001781.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McTaggart-Cowan, R., and Coauthors, 2022: Using stochastically perturbed parameterizations to represent model uncertainty. Part I: Implementation and parameter sensitivity. Mon. Wea. Rev., 150, 28292858, https://doi.org/10.1175/MWR-D-21-0315.1.

    • Search Google Scholar
    • Export Citation
  • Ollinaho, P., and Coauthors, 2017: Towards process-level representation of model uncertainties: Stochastically perturbed parameterizations in the ECMWF ensemble. Quart. J. Roy. Meteor. Soc., 143, 408422, https://doi.org/10.1002/qj.2931.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., 2001: A nonlinear dynamical perspective on model error: A proposal for non-local stochastic-dynamic parameterization in weather and climate prediction models. Quart. J. Roy. Meteor. Soc., 127, 279304, https://doi.org/10.1002/qj.49712757202.

    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., G. J. Shutts, and R. Swinbank, 1986: Alleviation of a systematic westerly bias in general circulation and numerical weather prediction models through an orographic gravity wave drag parameterization. Quart. J. Roy. Meteor. Soc., 112, 10011039, https://doi.org/10.1002/qj.49711247406.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., R. Buizza, F. Doblas-Reyes, T. Jung, M. Leutbecher, G. J. Shutts, M. Steinheimer, and A. Weisheimer, 2009: Stochastic parameterization and model uncertainty. European Centre for Medium-Range Weather Forecasts Tech. Rep. 598, 44 pp., https://www.ecmwf.int/sites/default/files/elibrary/2009/11577-stochastic-parametrization-and-model-uncertainty.pdf.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Park, S., and C. S. Bretherton, 2009: The University of Washington shallow convection and moist turbulence schemes and their impact on climate simulations with the community atmospheric model. J. Climate, 22, 34493469, https://doi.org/10.1175/2008JCLI2557.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Parrish, D. F., and J. C. Derber, 1992: The National Meteorological Center’s spectral statistical-interpolation analysis system. Mon. Wea. Rev., 120, 17471763, https://doi.org/10.1175/1520-0493(1992)120<1747:TNMCSS>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Patoine, A., and L. Separovic, 2021: Regional Ensemble Prediction System (REPS) version 4.0 summary of changes with respect to version 3.1 and validation. Environment and Climate Change Canada Tech. Note, 35 pp., https://collaboration.cmc.ec.gc.ca/cmc/cmoi/product_guide/docs/tech_notes/technote_reps-400_e.pdf.

    • Crossref
    • Export Citation
  • Pilon, R., C. Zhang, and J. Dudhia, 2016: Roles of deep and shallow convection and microphysics in the MJO simulated by the model for prediction across scales. J. Geophys. Res. Atmos., 121, 10 57510 600, https://doi.org/10.1002/2015JD024697.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Reynolds, C. A., J. A. Ridout, and J. G. McLay, 2011: Estimation of parameter variations in the U.S. Navy global ensemble. Tellus, 63, 841857, https://doi.org/10.1111/j.1600-0870.2011.00532.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rodwell, M. J., and Coauthors, 2013: Characteristics of occasionally poor medium-range weather forecasts for Europe. Bull. Amer. Meteor. Soc., 94, 13931405, https://doi.org/10.1175/BAMS-D-12-00099.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rodwell, M. J., S. T. K. Lang, N. B. Ingleby, N. Bormann, E. Hólm, F. Rabier, D. S. Richardson, and M. Yamaguchi, 2016: Reliability in ensemble data assimilation. Quart. J. Roy. Meteor. Soc., 142, 443454, https://doi.org/10.1002/qj.2663.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sanchez, C., K. D. Williams, and M. Collins, 2016: Improved stochastic physics schemes for global weather and climate models. Quart. J. Roy. Meteor. Soc., 142, 147159, https://doi.org/10.1002/qj.2640.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sandu, I., P. Bechtold, A. Beljaars, A. Bozzo, F. Pithan, T. G. Shepherd, and A. Zadra, 2016: Impacts of parameterized orographic drag on the Northern Hemisphere winter circulation. J. Adv. Model. Earth Syst., 8, 196211, https://doi.org/10.1002/2015MS000564.

    • Search Google Scholar
    • Export Citation
  • Shutts, G., 2005: A kinetic energy backscatter algorithm for use in ensemble prediction systems. Quart. J. Roy. Meteor. Soc., 131, 30793102, https://doi.org/10.1256/qj.04.106.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shutts, G., 2015: A stochastic convective backscatter scheme for us in ensemble prediction systems. Quart. J. Roy. Meteor. Soc., 141, 26022616, https://doi.org/10.1002/qj.2547.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shutts, G., and A. C. Pallarès, 2014: Assessing parameterization uncertainty associated with horizontal resolution in numerical weather prediction models. Philos. Trans. Roy. Soc., A372, 114, https://doi.org/10.1098/rsta.2013.0284.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Souders, M. B., B. A. Colle, and E. K. M. Chang, 2014: The climatology and characteristics of Rossby wave packets using a feature-based tracking technique. Mon. Wea. Rev., 142, 35283548, https://doi.org/10.1175/MWR-D-13-00371.1.

    • Search Google Scholar
    • Export Citation
  • Sundqvist, H., 1978: A parameterization scheme for non-convective condensation including prediction of cloud water content. Quart. J. Roy. Meteor. Soc., 104, 677690, https://doi.org/10.1002/qj.49710444110.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sundqvist, H., E. Berge, and J. E. Kristjánsson, 1989: Condensation and cloud parameterization studies with a mesoscale numerical weather prediction model. Mon. Wea. Rev., 117, 16411657, https://doi.org/10.1175/1520-0493(1989)117<1641:CACPSW>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sverdrup, H. U., M. W. Johnson, and R. H. Fleming, 1942: The Oceans. Prentice-Hall, 1087 pp.

  • Tilinina, N., A. Gavrikov, and S. K. Gulev, 2018: Association of North Atlantic surface turbulent heat fluxes with midlatitude cyclones. Mon. Wea. Rev., 146, 36913715, https://doi.org/10.1175/MWR-D-17-0291.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vaillancourt, P. A., and Coauthors, 2015: Improvements to the Regional Deterministic Prediction System (RDPS) version 2.0.0 to version 3.0.0. Environment and Climate Change Canada Doc., 78 pp., http://collaboration.cmc.ec.gc.ca/cmc/cmoi/product_guide/docs/lib/op_systems/doc_opchanges/technote_rdps300_20121003_e.pdf.

    • Crossref
    • Export Citation
  • Vannitsem, S., and Coauthors, 2021: Statistical postprocessing for weather forecasts: Review, challenges, and avenues in a big data world. Bull. Amer. Meteor. Soc., 102, E681E699, https://doi.org/10.1175/BAMS-D-19-0308.1.

    • Search Google Scholar
    • Export Citation
  • Vitart, F., 2017: Madden–Julian oscillation prediction and teleconnections in the S2S database. Quart. J. Roy. Meteor. Soc., 143, 22102220, https://doi.org/10.1002/qj.3079.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • von Salzen, K., N. A. McFarlane, and M. Lazare, 2005: The role of shallow convection in the water and energy cycles of the atmosphere. Climate Dyn., 25, 671688, https://doi.org/10.1007/s00382-005-0051-2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, J., J. Chen, J. Du, Y. Zhang, Y. Xia, and G. Deng, 2018: Sensitivity of ensemble forecast verification to model bias. Mon. Wea. Rev., 146, 781796, https://doi.org/10.1175/MWR-D-17-0223.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Weisheimer, A., S. Corti, T. Palmer, and F. Vitart, 2014: Addressing model error through atmospheric stochastic physical parameterizations: Impact on the coupled ECMWF seasonal forecasting system. Philos. Trans. Roy. Soc., A372, 121, https://doi.org/10.1098/rsta.2013.0290.

    • Search Google Scholar
    • Export Citation
  • Wirth, V., M. Riemer, E. K. M. Chang, and O. Martius, 2018: Rossby wave packets on the midlatitude waveguide—A review. Mon. Wea. Rev., 146, 19652001, https://doi.org/10.1175/MWR-D-16-0483.1.

    • Search Google Scholar
    • Export Citation
  • Wood, R., 2012: Stratocumulus clouds. Mon. Wea. Rev., 140, 23732423, https://doi.org/10.1175/MWR-D-11-00121.1.

  • Yanai, M., S. Esbensen, and J. H. Chu, 1973: Determination of bulk properties of tropical cloud clusters from large-scale heat and moisture budgets. J. Atmos. Sci., 30, 611627, https://doi.org/10.1175/1520-0469(1973)030<0611:DOBPOT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • View in gallery
    Fig. 1.

    Summary of fractional contributions Cf to ensemble spread in the (a)–(c) multiphysics, (d)–(f) SPPT, and (g)–(i) SPP ensembles in the (top) northern midlatitudes (25°–70°N); (middle) tropics (25°N–25°S); and (bottom) southern midlatitudes (25°–70°S). Each panel shows contributions to the 250-hPa zonal wind (U250; first panel row), 500-hPa geopotential height (Z500; second panel row), 850-hPa temperature (T850; third panel row), 850-hPa specific humidity (Q850; fourth panel row), 2-m temperature (T2m; fifth panel row), and sea level pressure (SLP; sixth panel row). Contributions are averaged over 5-day (pentad) periods as indicated along the abscissa, with color filling following the values indicated on the color bars and annotations for values that exceed the color bar extrema. The values of contributions that exceed 0.1% and are assessed to be significant at the 99% level using a 1000-member bootstrap test are shown explicitly on the plot.

  • View in gallery
    Fig. 2.

    Contributions to pentad-1 ensemble spread in the (a)–(d) multiphysics (MP), (e)–(h) SPPT, and (i)–(l) SPP ensembles for (top) U250, (top middle) Z500, (bottom middle) T850, and (bottom) Q850. Values are color shaded according to the values on the color bars at the bottom of each panel, with contributions that are assessed to be significant at the 99% level using a 1000-member bootstrap test indicated by dark shading as indicated on the lower color bar, whereas those that fail to meet this threshold are plotted with the lighter colors of the upper color bar. Regions in which the pressure surface intersects the ground are masked with light-green shading. The tropical Pacific region used later in Figs. 4 and 5 is outlined in black on 850-hPa plots.

  • View in gallery
    Fig. 3.

    Contributions to ensemble spread as in Fig. 2, but for near-surface fields, showing (a),(c),(e) screen-level temperature and (b),(d),(f) sea level pressure. Regions in which surface pressure is less than 900 hPa are masked with green shading to avoid excessive subterranean extrapolation.

  • View in gallery
    Fig. 4.

    Multiphysics ensemble temperature (abscissa; °C) and specific humidity (ordinate; g kg−1) phase plane at 850 hPa over the tropical Pacific Ocean (Fig. 2). Four subensembles are plotted with different colors centered on cyan (multiphysics designation in Table 3) as shown in the legend, classified according to the turbulent mixing length (local or nonlocal following Table 1) and moist physics package (mass-flux or moisture convergence, abbreviated here as “Moist. Conv.”). Dots indicate the mean departures for each subensemble from the full ensemble mean, each corresponding to the pentad-1 average departure for a single initialization. A kernel density estimate for the individual members is shown in contours (normalized densities of 1, 2, 4, 8, and 16). Dashed gray lines indicate the origin, which corresponds to the mean of the full ensemble. Annotations in light gray facilitate physical process discussions; labels for the Prandtl number Pr, although similar for all subensembles, are provided only for the nonlocal + moisture convergence group in the lower-right quadrant. The dotted gray box centered on the origin represents the subplane plotted in Fig. 5, below.

  • View in gallery
    Fig. 5.

    Phase plane of 850-hPa temperature and moisture over the tropical Pacific Ocean as in Fig. 4, but for ensembles using the (a) SPPT and (b) SPP schemes on a subplane near the origin (dotted gray box in Fig. 4). Gray points show temperature and moisture departures from the ensemble mean for all members and cases, summarized using kernel density estimates in solid-contours experiment-specific color coding according to Table 3. Also shown in each panel is the density estimate for the departure of the control member from the ensemble mean (black contours) and the best-fit linear model (experiment-colored thin dashed line, both with nonzero slopes that are significant at the 99% level according to an F-test of the regression against an intercept-only model).

  • View in gallery
    Fig. 6.

    Summary of Cf from the (a)–(c) multiphysics, (d)–(f) SPPT, and (g)–(i) SPP schemes to the fCRPS. Plotting conventions follow those used in Fig. 1.

  • View in gallery
    Fig. 7.

    Contributions of the (a)–(d) multiphysics, (e)–(h) SPPT, and (i)–(l) SPP schemes to the upper-air fCRPS. Plotting conventions follow those used in Fig. 2. The North Atlantic region used in Figs. 11 and 12 is outlined in black on 500-hPa heights and 850-hPa temperatures. Also outlined in black for 500-hPa heights is the Maritime Continent region used later in Fig. 10 and Table 4.

  • View in gallery
    Fig. 8.

    Contribution of the (a),(b) multiphysics; (c),(d) SPPT; and (e),(f) SPP schemes to the near-surface fCRPS. Plotting conventions follow those used in Fig. 3. The Maritime Continent region used later in Fig. 10 and Table 4 is outlined in black on screen-level temperatures.

  • View in gallery
    Fig. 9.

    Spread–reliability diagrams (Leutbecher et al. 2007) for 850-hPa temperature in 72-h forecasts over the (a),(c) northern midlatitudes and (b),(d) tropics for (left) the multiphysics configuration and (right) the stochastic schemes. Naming and color coding follow Table 3, with integrations grouped by their use of the updated physics suite so that each experiment is directly compared with the appropriate control ensemble. The climatological standard deviations used to normalize the spread (abscissa) and error (ordinate) values are computed for the 1989–2016 period from the ERA-Interim reanalysis (Dee et al. 2011). The means of the 20 percentile-based spread bins are plotted with circles, and a color-coded square is placed at the overall mean of each distribution.

  • View in gallery
    Fig. 10.

    Cumulative distribution functions for pentad-1 (a) 500-hPa geopotential heights and (b) screen-level temperatures averaged over the Maritime Continent region shown in Fig. 7 in the ensembles indicated on the legend (Table 3). The multiphysics-based results are divided into subsets that are based on member configurations for the saltwater correction (“Salt = T” if the correction is active and “Salt = F” if it is not) and the value of the Prandtl number, with gray shading used to represent the subensembles as shown in the legend in the lower-right corner of the plot. The median for each ensemble and subensemble is shown at the bottom of the panel, with shading to indicate the 95% confidence interval based on a 1000-member bootstrap. The medians are plotted on separate rows for readability, with dashed lines connecting the full ensemble medians to their values within the distributions (intersection with the 0.5 level of the cumulative relevant distribution function).

  • View in gallery
    Fig. 11.

    Relationship between pentad-1 bias (abscissa) and MAE (ordinate) over the North Atlantic region (Fig. 7) for (a),(c),(e) 500-hPa height and (b),(d),(f) 850-hPa temperature in ensembles using the (left) multiphysics configuration; (center) SPPT; and (right) SPP schemes. Kernel density estimates are plotted for each experiment in contours that are color coded according to Table 3, normalized by sample size and plotted at unit density intervals from 1 to 5. Also shown are the distributions of the deterministic control member (thin black contours) and of the control ensemble (color shading) in which the relevant model error representation is absent (Table 3). The multiphysics ensemble is decomposed into subensembles (solid and dashed contours) distinguished by their treatment of the roughness length for scalars, which directly affects heat fluxes over water (Table 1).

  • View in gallery
    Fig. 12.

    Spread–reliability diagrams for 72-h forecasts of (a),(c) 500-hPa height and (b),(d) 850-hPa temperature over the North Atlantic region (Fig. 7) in ensembles using (left) the multiphysics configuration and (right) the stochastic schemes. Plotting follows the conventions described in Fig. 9.

  • View in gallery
    Fig. 13.

    Summary statistics of the “step 1” transition from the combined SPPT+multiphysics approach to the SPP scheme, shown as the change in Cf to (a)–(c) ensemble spread and (d)–(f) fCRPS. To eliminate the effects of the differing controls, this sensitivity is diagnosed using Cf differences rather than the contributions themselves (Table 5); however, the plotting conventions follow those employed for Fig. 1. Annotations with arrows indicate the interpretation of sign of Cf as a reference.

  • View in gallery
    Fig. 14.

    Summary statistics following the “step 2” addition of the updated physics suite. As shown in Table 5, this sensitivity is computed using the SPP-based ensemble as “experiment” and the combined SPPT+multiphysics ensemble as “control” in Eq. (1). Plotting conventions follow those employed for Fig. 1.

  • View in gallery
    Fig. 15.

    Results of the “step 3” full GEPS upgrade, including all external changes to the system. As shown in Table 5, this sensitivity is computed by using the updated system as experiment and the current operational system as control in Eq. (1). Blue shading is indicative of forecast skill improvement. Plotting conventions follow those employed for Fig. 1.

  • View in gallery
    Fig. 16.

    Global ensemble spread (dashed) and ensemble mean unbiased RMSEu (solid) growth in the current operational GEPS (blue) and updated GEPS configuration (red) for (a) 250-hPa zonal winds, (b) 500-hPa height, and (c) 850-hPa temperatures. The vertical dashed gray lines indicate the 72-h forecast time used for the spread–reliability diagrams in Fig. 17, below.

  • View in gallery
    Fig. 17.

    Global spread–reliability diagrams for 72-h forecasts from the current operational GEPS (blue) and updated GEPS configuration (red) for (a) 250-hPa zonal winds, (b) 500-hPa heights, and (c) 850-hPa temperatures. Plotting follows the conventions described in Fig. 9.

All Time Past Year Past 30 Days
Abstract Views 28 28 0
Full Text Views 597 597 33
PDF Downloads 370 370 35

Using Stochastically Perturbed Parameterizations to Represent Model Uncertainty. Part II: Comparison with Existing Techniques in an Operational Ensemble

Ron McTaggart-CowanaAtmospheric Numerical Weather Prediction Research Section, Environment and Climate Change Canada, Dorval, Quebec, Canada

Search for other papers by Ron McTaggart-Cowan in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0002-3092-4365
,
Leo SeparovicaAtmospheric Numerical Weather Prediction Research Section, Environment and Climate Change Canada, Dorval, Quebec, Canada

Search for other papers by Leo Separovic in
Current site
Google Scholar
PubMed
Close
,
Martin CharronaAtmospheric Numerical Weather Prediction Research Section, Environment and Climate Change Canada, Dorval, Quebec, Canada

Search for other papers by Martin Charron in
Current site
Google Scholar
PubMed
Close
,
Xingxiu DengbNumerical Weather Prediction Development Section, Meteorological Service of Canada, Dorval, Quebec, Canada

Search for other papers by Xingxiu Deng in
Current site
Google Scholar
PubMed
Close
,
Normand GagnonbNumerical Weather Prediction Development Section, Meteorological Service of Canada, Dorval, Quebec, Canada

Search for other papers by Normand Gagnon in
Current site
Google Scholar
PubMed
Close
,
Pieter L. HoutekamercData Assimilation Research Section, Environment and Climate Change Canada, Dorval, Quebec, Canada

Search for other papers by Pieter L. Houtekamer in
Current site
Google Scholar
PubMed
Close
, and
Alain PatoinebNumerical Weather Prediction Development Section, Meteorological Service of Canada, Dorval, Quebec, Canada

Search for other papers by Alain Patoine in
Current site
Google Scholar
PubMed
Close
Open access

Abstract

The ability of a stochastically perturbed parameterization (SPP) approach to represent uncertainties in the model component of the Canadian Global Ensemble Prediction System was demonstrated in Part I of this investigation. The goal of this second step in SPP evaluation is to determine whether the scheme represents a viable alternative to the current operational combination of a multiphysics configuration and stochastically perturbed parameterization tendencies (SPPT). An assessment of the impact of each model uncertainty estimate in isolation reveals that, although the multiphysics configuration is highly effective at generating ensemble spread, it is often the result of differing biases rather than a reflection of flow-dependent error growth. Moreover, some of the members of the multiphysics ensemble suffer from large errors on regional scales as a result of suboptimal configurations. The SPP scheme generates a greater diversity of member solutions than the SPPT scheme in isolation, and it has an impact on forecast performance that is similar to that of current operational uncertainty estimates. When the SPP framework is combined with recent upgrades to the model physics suite that are only applicable in the stochastic perturbation context, the quality of global ensemble guidance is significantly improved.

Significance Statement

The stochastically perturbed parameterization (SPP) technique was introduced in Part I to represent model uncertainties in forecasts generated by an operational global ensemble prediction system. We focus here on the viability of this technique as a replacement for the system’s current uncertainty estimates: multiphysics and stochastic perturbations of physics tendencies. Despite the practical success of this combination, it suffers from physical inconsistencies and poor conservation properties. The adoption of SPP allows the ensemble to benefit from a recent set of model updates that couple with this new representation of model uncertainty to yield significant improvements in the quality of forecasts generated by the system.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Ron McTaggart-Cowan, ron.mctaggart-cowan@canada.ca

Abstract

The ability of a stochastically perturbed parameterization (SPP) approach to represent uncertainties in the model component of the Canadian Global Ensemble Prediction System was demonstrated in Part I of this investigation. The goal of this second step in SPP evaluation is to determine whether the scheme represents a viable alternative to the current operational combination of a multiphysics configuration and stochastically perturbed parameterization tendencies (SPPT). An assessment of the impact of each model uncertainty estimate in isolation reveals that, although the multiphysics configuration is highly effective at generating ensemble spread, it is often the result of differing biases rather than a reflection of flow-dependent error growth. Moreover, some of the members of the multiphysics ensemble suffer from large errors on regional scales as a result of suboptimal configurations. The SPP scheme generates a greater diversity of member solutions than the SPPT scheme in isolation, and it has an impact on forecast performance that is similar to that of current operational uncertainty estimates. When the SPP framework is combined with recent upgrades to the model physics suite that are only applicable in the stochastic perturbation context, the quality of global ensemble guidance is significantly improved.

Significance Statement

The stochastically perturbed parameterization (SPP) technique was introduced in Part I to represent model uncertainties in forecasts generated by an operational global ensemble prediction system. We focus here on the viability of this technique as a replacement for the system’s current uncertainty estimates: multiphysics and stochastic perturbations of physics tendencies. Despite the practical success of this combination, it suffers from physical inconsistencies and poor conservation properties. The adoption of SPP allows the ensemble to benefit from a recent set of model updates that couple with this new representation of model uncertainty to yield significant improvements in the quality of forecasts generated by the system.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Ron McTaggart-Cowan, ron.mctaggart-cowan@canada.ca

1. Introduction

Ensemble NWP systems should provide reliable estimates of the potential for errors in the guidance that they generate. To do this, they need to account for the impact of myriad uncertainties on forecast skill. One of the leading sources of possible forecast error is the model itself, which suffers from formulation approximations, discretization errors, and incomplete representations of physical processes. In McTaggart-Cowan et al. 2022, hereinafter Part I) a stochastically perturbed parameterizations (SPP) scheme was implemented in the Canadian Global Ensemble Prediction System (GEPS) to represent model error. The SPP strategy was found to produce a broad diversity of member solutions through perturbations to specific atmospheric processes. In this second part of the investigation, results from the SPP-based system are compared with those generated by ensembles using well-established GEPS model error depictions to assess the viability of adopting SPP in an operational context.

The GEPS has used a combination of three distinct model-based uncertainty estimates to generate reliable ensemble predictions since 2007 (Houtekamer et al. 2009; Charron et al. 2010).1 Its multiphysics implementation involves the use of member-specific physical parameterization configurations (Houtekamer et al. 1996; Houtekamer 2011). Each member’s physics tendencies are also perturbed using a variant of the stochastically perturbed parameterization tendency (SPPT) scheme (Buizza et al. 1999; Palmer et al. 2009; Charron et al. 2010). A stochastic kinetic energy backscatter scheme seeds upscale error growth in regions with large dissipative energy sinks (Shutts 2005; Charron et al. 2010). This study focuses on the first two of these uncertainty estimates (multiphysics and SPPT) because of their conceptual overlap with the SPP framework.

Multiphysics ensemble configurations are designed to sample broadly from uncertainties related to the free parameters, closures and formulations used to represent physical processes (Berner et al. 2011, 2015; Houtekamer 2011). This makes them highly effective at generating ensemble spread while ensuring that individual members are internally consistent and conserve relevant properties to the level achieved by the schemes themselves (Jankov et al. 2019). Moreover, the restricted number of available schemes makes rigorous optimization of multiphysics-based systems a tractable problem (Lee et al. 2012; Jeworrek et al. 2021).

Despite its effectiveness, systematic differences between the members of multiphysics-based systems create undesirable heterogeneity in forecast quality and biases across the ensemble (Berner et al. 2015; Kalina et al. 2021). Minimizing inter-member skill differences by ensuring uniform deterministic guidance quality is a costly and time-consuming undertaking (Berner et al. 2011; Reynolds et al. 2011). Member-specific model climates also create multimodal distributions, a significant problem for ensemble data assimilation systems (Bocquet et al. 2010). The associated spread serves to increase the likelihood that the forecast envelope encompasses the observed state; however, systematic inter-member differences do not accurately represent flow-dependent uncertainty (Berner et al. 2015) and may degrade the sensitivity of the system (Cui et al. 2012; Wang et al. 2018). Although most calibration schemes can minimize the impact of this problem on postprocessed ensemble statistics (Vannitsem et al. 2021), techniques that mitigate the effects of biases on atmospheric circulations and physics during the integration are still under development (Chen et al. 2020; Krinner et al. 2020; Ayar et al. 2021). Because these process-level sources of uncertainty are of primary interest here, direct model outputs are used throughout this investigation.

The SPPT scheme is employed in most operational NWP ensembles, a testament to its ability to generate skillful predictions (Hou et al. 2006; Palmer et al. 2009; Sanchez et al. 2016; Leutbecher et al. 2017). Its success stems from the fact that leading-order errors persist both in our understanding of physical processes and in our ability to develop parameterizations that accurately depict them (Sandu et al. 2016; Christensen 2020). As a result, the tendencies generated by the model’s physical parameterization suite contain significant uncertainty (Buizza et al. 1999), an error source that can dramatically impact forecast quality (Rodwell et al. 2013; Lillo and Parsons 2017; Baumgart et al. 2019).

Despite the practical success of the SPPT scheme, it suffers from potentially important conceptual limitations. Multiplicative rescaling of full physics tendencies means that perturbations will be small in regions with small net tendencies regardless of whether there is active cancellation between individual schemes (Shutts and Pallarès 2014; Christensen et al. 2017). Conversely, perturbations can be excessive over layers in which tendencies are large but well constrained (Lock et al. 2019). The posthoc application of perturbations can also lead to physically unrealistic solutions (Leutbecher et al. 2017) and disrupt the conservation of momentum, energy and total water, potentially generating errors in the model climate (Palmer 2001; Sanchez et al. 2016; Lang et al. 2021).

The SPP approach to model uncertainty representation, proposed by Ollinaho et al. (2017) and Jankov et al. (2017), was introduced in detail in Part I. Because it involves transient perturbations to free parameters and closures (hereinafter referred to as “elements”) within the model, physical consistency and conservation properties are ensured and the model climate remains unaffected within the bounds of noise-induced drift (Berner et al. 2017; Lang et al. 2021). The SPP technique is also conceptually appealing because it represents the origins of errors close to their sources within the model. Although initial implementations of the SPP scheme have struggled to produce a sufficiently diverse set of member solutions (Ollinaho et al. 2017; Jankov et al. 2019; Frogner et al. 2022), recent refinements suggest that the technique may be capable of sampling uncertainty as effectively as the SPPT scheme (Kalina et al. 2021; Lang et al. 2021).

The purpose of this study is to assess the potential for transition to an SPP-based ensemble in the operational GEPS. This study begins with a brief description of the system and the model uncertainty representations that it is equipped to employ (section 2). The characteristics of the guidance generated by configurations using each of these schemes in isolation are documented in section 3. Beginning with the transition from a combination of multiphysics and SPPT to an SPP-based representation of model uncertainty, the components of a major update to the GEPS are assembled incrementally in section 4, leading to the concluding discussion in section 5.

2. Data and methods

This section begins with a brief introduction to the Global Environmental Multiscale (GEM) model and the adopted experimental design (section 2a). This is followed by an overview of the multiphysics and SPPT implementations in the GEPS (sections 2b and 2c, respectively), important elements of which have not appeared previously in the literature. This background material complements the detailed description of the SPP scheme provided in Part I.

a. Model and experimental design

The GEM model used in this study was described in section 2 of Part I. Of particular relevance here is a recent update to the model’s suite of physical parameterizations (McTaggart-Cowan et al. 2019), which benefits from increased vertical resolution and entails the removal of a large set of outdated schemes. Many of these parameterizations were used in the multiphysics configuration of the GEPS, making it impossible to use this approach in the modernized-physics context. Conversely, the SPP scheme is only implemented within the updated physics package. Two sets of control ensembles are therefore needed, each run with a stochastic kinetic energy backscatter scheme (Charron et al. 2010) as the only model uncertainty estimate.2 The impact of the multiphysics technique is evaluated using the older physics configuration, while performance of the stochastic schemes is assessed within the updated-physics framework. A crossover experiment in which the SPPT scheme is activated in the “old physics” suite confirms that sensitivities are similar in the two systems, indicating that comparisons of the marginal impacts of uncertainty estimates are robust across model versions (section 1 of the online supplemental material).

The GEPS used in this study mirrors the 20-member configuration (plus 1 control member run without initial or model perturbations) described in Part I. A total of 44 initializations of 15-day ensemble forecasts are made at 36-h intervals over January–February 2020. Boreal summer integrations (July–August 2019) show that seasonality has a leading-order influence on the structure of the sensitivities assessed here (section 2 of the online supplemental material). As a result, the Northern Hemisphere is also referred to as the “winter hemisphere” and the Southern Hemisphere as the “summer hemisphere” in section 3 as appropriate to enhance the generality of the analyses and conclusions.

The ensemble sensitivity formalism used to assess the impact of model error representations was introduced in Part I. The contribution of a scheme to a forecast metric J at a given lead time is assessed as ΔS = JexperimentJcontrol. This quantity is diagnosed from an ensemble that uses a single uncertainty estimate (subscript “experiment”) and its control ensemble (subscript “control”) in which all relevant schemes are deactivated.3 Following Reynolds et al. (2011), the fractional contribution Cf of each scheme is computed as follows:
Cf(J)=JexperimentJcontrolJcontrol,
a dimensionless quantity that quantifies the marginal impact of a specific scheme in a form that facilitates comparison across forecast fields and lead times.

The evaluation strategy adopted in this study follows Part I, with GEPS results as compared with analyses from the Canadian Global Deterministic Prediction System (Buehner et al. 2015). Pentad averaging is used to minimize the volume of displayed data while enhancing the statistical stability of the analysis.

b. The GEPS multiphysics configuration

The wide variety of algorithms used to represent physical processes in the atmosphere is indicative of the uncertainty associated with any individual depiction thereof. The multiphysics approach to model error estimation leverages this diversity to create divergent solutions in regions where the differences between parameterizations have a meaningful impact on the local atmospheric state.

Each member in the operational GEPS has a unique mix of physical parameterizations and values for key free parameters within the schemes (Houtekamer 2011; current configurations are listed in Table 1). This strategy is consistent with the Monte Carlo approach to ensemble design adopted at the CMC (Buizza et al. 2005) and has the benefit of broadly sampling formulation-level uncertainties.

Table 1

Multiphysics configuration used in the existing operational global NWP ensemble. The configuration of the control member is shown in boldface type, with the 20 perturbed members made up of uniformly sampled combinations of these physics options.

Table 1

The multiphysics approach creates important practical challenges for model developers. Designing, implementing and maintaining a selection of equally skillful physical parameterizations represents a significant investment, particularly if these schemes need to be interoperable. Such a “plug and play” requirement also represents a departure from the current emphasis on tighter integration between schemes (Gross et al. 2016). As a result, the moisture-convergence-based moist physics schemes (Table 1) were removed from the model as part of its recent upgrade (McTaggart-Cowan et al. 2019), a change that precludes the use of the multiphysics approach within the updated parameterization suite. Despite the success of the multiphysics technique in Canadian ensembles, the accumulating list of limitations associated with this strategy suggests that a transition to alternative forms of model uncertainty representation should be considered.

c. Stochastic perturbation of physical tendencies in the GEPS

The observation that uncertainties associated with unresolved and diabatic processes are an important source of model error led Buizza et al. (1999) to propose a model uncertainty estimate that relies on perturbations to state-variable tendencies generated by the physical parameterization suite. The SPPT strategy is highly effective at generating ensemble spread in sensitive regions with significant net tendencies (Christensen et al. 2015).

The net physical temperature T and horizontal wind V tendencies in each member of the operational GEPS are perturbed [denoted with ()′] multiplicatively at each time step as follows:
(dxdt)=(1+ΛF)dxdt|phy,
for x ∈ {Vh, T}, where F is a random number in the range [−0.5, 0.5] and Λ is a prescribed SPPT modulation factor. Unlike most operational SPPT implementations, moisture remains unperturbed to limit the magnitude of perturbation-induced supersaturation. The distribution of F is determined by the stochastic pattern generator described in section 3a of Part I. Based on first-order autoregressive processes, the structure of F varies in time and space based on the parameters shown in Table 2.
Table 2

Parameters used in stochastic field generation for the SPPT scheme. Further definition of these parameters can be found in the appendix of Part I.

Table 2

The strength of perturbations is modulated in the vertical using a profile of Λ coefficients to prevent instabilities associated with strong tendencies in the boundary layer (Leutbecher et al. 2017). In the GEPS, Λ is a step function at a sigma coordinate value of 0.99 (near 100 m AGL) that transitions from 0 near the surface to 1 aloft. To avoid instabilities induced by large deep-convective tendencies, the full profile of Λ is set to zero when this scheme is active (Gagnon et al. 2013).

3. Comparison of model uncertainty representations

The behavior of three distinct representations of model uncertainty will be assessed in isolation in this section: multiphysics, SPPT and SPP. The schemes’ abilities to generate ensemble spread are assessed in section 3a, while their impacts on guidance quality are evaluated in section 3b. Although the results shown here are system-specific, the widespread use of similar strategies suggests that the conclusions that they lead to may be applicable more generally. A description of the GEPS configurations used in this section can be found in Table 3.

Table 3

Description of GEPS configurations used in section 3. The “Experiment” and “Control” columns refer to the corresponding subscripts in Eq. (1). The term “no error scheme” is used here to describe the control configurations in which perturbations to the initial conditions remain but the only estimate of model uncertainty comes from the stochastic kinetic energy backscatter scheme (common to all configurations in this table).

Table 3

a. Growth and structure of ensemble spread

A fundamental requirement for well-balanced ensembles is that they generate a range of solutions whose diversity matches forecast error. This ensemble spread is defined as the square root of the mean ensemble variance [Eq. (5) of Part I]. The fractional contribution [Cf in Eq. (1)] of each model uncertainty representation to ensemble spread is shown in Fig. 1. All schemes contribute to ensemble spread most significantly in pentads 1 and 2, with the saturation of midlatitude spread diminishing impacts at longer leads. In the tropics, however, the potential for model uncertainty-induced spread growth throughout the forecast period is demonstrated by persistently large Cf values.

Fig. 1.
Fig. 1.

Summary of fractional contributions Cf to ensemble spread in the (a)–(c) multiphysics, (d)–(f) SPPT, and (g)–(i) SPP ensembles in the (top) northern midlatitudes (25°–70°N); (middle) tropics (25°N–25°S); and (bottom) southern midlatitudes (25°–70°S). Each panel shows contributions to the 250-hPa zonal wind (U250; first panel row), 500-hPa geopotential height (Z500; second panel row), 850-hPa temperature (T850; third panel row), 850-hPa specific humidity (Q850; fourth panel row), 2-m temperature (T2m; fifth panel row), and sea level pressure (SLP; sixth panel row). Contributions are averaged over 5-day (pentad) periods as indicated along the abscissa, with color filling following the values indicated on the color bars and annotations for values that exceed the color bar extrema. The values of contributions that exceed 0.1% and are assessed to be significant at the 99% level using a 1000-member bootstrap test are shown explicitly on the plot.

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

In the northern midlatitudes (winter; Figs. 1a,d,g), the multiphysics configuration contributes significantly to ensemble spread in the lower troposphere, particularly for 850-hPa humidity (Fig. 1a). The stochastic schemes have a larger impact on upper-air spread early in the forecast period (Figs. 1d and 1g). The SPP scheme also generates more spread in long-range forecasts than the other techniques and produces the largest spread in screen-level temperatures (Fig. 1g).

Spread contributions in the southern midlatitudes (summer; Figs. 1c,f,i) generally mirror those of their winter counterparts but with a reduced amplitude. This asymmetry is largely seasonal, with the stochastic schemes generating diverse solutions in the presence of enhanced baroclinic instability and associated Rossby wave activity (Souders et al. 2014).

The SPP scheme is the most effective at generating ensemble spread in the tropics, with contributions that persist throughout the forecast period (Fig. 1h). Neither the multiphysics configuration nor the SPPT scheme generates significant tropical spread in pentad 3 (Figs. 1b and 1e). Even at shorter range, a conservative selection of surface layer multiphysics perturbations limits the diversity of screen-level temperatures across the ensemble (Fig. 1b). Meanwhile, the lack of humidity perturbation in the SPPT implementation and its deactivation under deep-convective conditions renders the scheme ineffective at generating moisture spread at the top of the boundary layer (Fig. 1e). An important conclusion to be drawn from Fig. 1 is that the SPP scheme is capable of generating ensemble spread that is comparable to or exceeds that of either the SPPT or the multiphysics approaches in isolation.

All of the model uncertainty representations considered here focus primarily on error contributions from physical processes that are unevenly distributed around the globe. Moreover, the growth rates of the induced perturbations depend on the local background state. There is therefore considerable spatial variability in the ensemble spread generated by the schemes (Figs. 2 and 3). Although the patterns that appear in these figures remain similar throughout the forecast period, the differences between the schemes are most stark in pentad 1 when responses to the perturbations are primarily direct and local.

Fig. 2.
Fig. 2.

Contributions to pentad-1 ensemble spread in the (a)–(d) multiphysics (MP), (e)–(h) SPPT, and (i)–(l) SPP ensembles for (top) U250, (top middle) Z500, (bottom middle) T850, and (bottom) Q850. Values are color shaded according to the values on the color bars at the bottom of each panel, with contributions that are assessed to be significant at the 99% level using a 1000-member bootstrap test indicated by dark shading as indicated on the lower color bar, whereas those that fail to meet this threshold are plotted with the lighter colors of the upper color bar. Regions in which the pressure surface intersects the ground are masked with light-green shading. The tropical Pacific region used later in Figs. 4 and 5 is outlined in black on 850-hPa plots.

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

Fig. 3.
Fig. 3.

Contributions to ensemble spread as in Fig. 2, but for near-surface fields, showing (a),(c),(e) screen-level temperature and (b),(d),(f) sea level pressure. Regions in which surface pressure is less than 900 hPa are masked with green shading to avoid excessive subterranean extrapolation.

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

The free-tropospheric ensemble spread generated by the multiphysics configuration is almost entirely restricted to the tropics (Figs. 2a–d), with large variability evident at the top of the boundary layer [to be investigated in more detail in section 3a(1)]. In contrast, SPPT-induced spread is focused along the midlatitude waveguides (Figs. 2e–h), a pattern that suggests that the induced perturbations affect the development and propagation of Rossby waves (Wirth et al. 2018). Although the SPP scheme generates variability across a range of latitudes (Figs. 2i–l), its influence is primarily focused on the Northern Hemisphere (winter): additional SPP elements may be required to sample uncertainty more comprehensively under midlatitude summer conditions.

The three model uncertainty representations induce distinct near-surface spread patterns (Fig. 3). The SPP-based perturbation of turbulent exchange coefficients (Part I) enhances screen-level temperature spread more effectively than the height-limited SPPT implementation (Figs. 3a,c,e). The widespread SPP-induced sea level pressure spread is structurally similar to that generated by SPPT in the winter midlatitudes but extends across the tropics (Figs. 3b,d,f). This increased variability is linked to exchange coefficient and advection perturbations (Part I), elements that do not appear to have functional analogs in the multiphysics or SPPT schemes.

Case study: Tropical Pacific boundary layer

The most striking difference between the three representations of model uncertainty appears at the top of the boundary layer over the subtropical oceans (Fig. 2). The stochastic schemes generate minimal ensemble spread while the multiphysics configuration produces large spread equatorward of the subtropical anticyclones.

As the trade wind inversion weakens in the central ocean basins, turbulent entrainment and trade wind cumulus transports begin to exert a leading influence on lower-tropospheric profiles (Bechtold et al. 2014). However, mixing remains suppressed in members of the multiphysics configuration that use the nonlocal form of the turbulent mixing length in their boundary layer schemes (Table 1), leading to a warmer and drier free atmosphere relative to those that employ the local estimate (separation subensembles along a negatively sloped axis in Fig. 4). Members that combine the nonlocal mixing length with the moisture convergence-based moist physics package display large departures from the ensemble mean, indicative of the entrainment reductions expected when the effects of shallow convection are not parameterized in the model (von Salzen et al. 2005; Park and Bretherton 2009; Pilon et al. 2016). This sensitivity is reduced in members that use the local mixing length estimate (smaller separation between the relevant subensembles in Fig. 4) because moist physics-induced changes in boundary layer structure do not directly affect turbulent transports over a deep layer.

Fig. 4.
Fig. 4.

Multiphysics ensemble temperature (abscissa; °C) and specific humidity (ordinate; g kg−1) phase plane at 850 hPa over the tropical Pacific Ocean (Fig. 2). Four subensembles are plotted with different colors centered on cyan (multiphysics designation in Table 3) as shown in the legend, classified according to the turbulent mixing length (local or nonlocal following Table 1) and moist physics package (mass-flux or moisture convergence, abbreviated here as “Moist. Conv.”). Dots indicate the mean departures for each subensemble from the full ensemble mean, each corresponding to the pentad-1 average departure for a single initialization. A kernel density estimate for the individual members is shown in contours (normalized densities of 1, 2, 4, 8, and 16). Dashed gray lines indicate the origin, which corresponds to the mean of the full ensemble. Annotations in light gray facilitate physical process discussions; labels for the Prandtl number Pr, although similar for all subensembles, are provided only for the nonlocal + moisture convergence group in the lower-right quadrant. The dotted gray box centered on the origin represents the subplane plotted in Fig. 5, below.

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

Modulation of surface fluxes by the Prandtl number Pr (Table 1) generates distinct clusters within each subensemble in Fig. 4. These clusters are separated along a positively tilted axis as heat and moisture fluxes vary in unison. The resulting changes to the near-surface profile affect turbulent transports over a deep layer when the nonlocal mixing length is used, leading to particularly large Pr-based cluster separation in these subensembles.

The process-induced heterogeneity in the temperature-moisture phase plane (Fig. 4) leads to large spread in the multiphysics ensemble; however, the diversity of solutions is primarily the result of distinct member climates rather than flow-dependent uncertainty. The non-Gaussianity of the resulting multimodal distributions also complicates the use and interpretation of multiphysics-based forecasts (Bertossa et al. 2021).

Conversely, the stochastic schemes generate unimodal temperature and moisture distributions over the tropical Pacific Ocean (Fig. 5). The limited ensemble spread generated by the SPPT scheme (Figs. 2g,h) is evident in the spatially limited distribution shown in Fig. 5a. This is inconsistent with the excessive SPPT-induced spread in this region identified by Rodwell et al. (2016), likely because of unperturbed GEPS moisture tendencies (section 2c). A slight negative slope in the distribution suggests that SPPT perturbations to mixing tendencies are at least partially responsible for spread generation. The SPP scheme leads to a broader range of moisture values (Fig. 5b), although variability remains much smaller than that predicted by the multiphysics configuration (Figs. 2d and 2l). The positive correlation between moisture and temperature tendencies shown in Fig. 5b implies that surface flux perturbations actively promote spread in the SPP-based ensemble.

Fig. 5.
Fig. 5.

Phase plane of 850-hPa temperature and moisture over the tropical Pacific Ocean as in Fig. 4, but for ensembles using the (a) SPPT and (b) SPP schemes on a subplane near the origin (dotted gray box in Fig. 4). Gray points show temperature and moisture departures from the ensemble mean for all members and cases, summarized using kernel density estimates in solid-contours experiment-specific color coding according to Table 3. Also shown in each panel is the density estimate for the departure of the control member from the ensemble mean (black contours) and the best-fit linear model (experiment-colored thin dashed line, both with nonzero slopes that are significant at the 99% level according to an F-test of the regression against an intercept-only model).

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

Differences between the control member and ensemble means in Fig. 5 suggest that a nonlinear response to perturbations produces systematic changes in perturbed-member states (Berner et al. 2017; Lang et al. 2021). Such sensitivities complicate efforts to minimize biases in the system. Further investigation of this behavior will help to identify SPP element formulations that limit these departures.

b. Growth and structure of ensemble forecast error

The impact of each model uncertainty representation on ensemble forecast skill needs to be assessed to ensure that the spread increases diagnosed above do not degrade individual member solutions sufficiently to negatively affect predictions. For this purpose, the fair continuous rank probability score (fCRPS) provides a probabilistic measure of ensemble performance that accounts for the limited ensemble size and spread differences between experiments (Ferro et al. 2008; Fricker et al. 2013). As in Part I:
fCRPS[(xj)M,y]=1Mj=1M|xjy|12M(M1)j=1Mk=1M|xjxk|,
for an M-member ensemble and observed value y. The fractional contribution Cf of each scheme to forecast skill is evaluated by using J = fCRPS in Eq. (1). The fCRPS is truly fair for the SPPT- and SPP-based ensembles because their members are exchangeable. Systematic inter-member differences in the multiphysics configuration violate this premise; however, the fCRPS remains an effective evaluation metric in practice (Leutbecher 2019).

All three model uncertainty representations generally lead to fCRPS reductions indicative of ensemble forecast skill improvements (Fig. 6). In the northern midlatitudes (winter), the multiphysics configuration improves upper-tropospheric forecast skill more effectively than the stochastic schemes (Figs. 6a,d,g). In the lower troposphere, however, the SPP scheme successfully improves forecasts throughout the long range in both hemispheres (Fig. 6g). The multiphysics configuration also elicits a persistent response in the southern extratropics (summer): a significant degradation in temperature predictions (Fig. 6c).

Fig. 6.
Fig. 6.

Summary of Cf from the (a)–(c) multiphysics, (d)–(f) SPPT, and (g)–(i) SPP schemes to the fCRPS. Plotting conventions follow those used in Fig. 1.

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

All schemes improve ensemble forecast skill in the tropics (Figs. 6b,e,h); however, the amplitude and longevity of fCPRS reductions is largest in the SPP and multiphysics configurations. The relatively limited contribution of the SPPT scheme is consistent both with the dominance of diabatic equilibrium across the tropics and with the lack of perturbations in the presence of deep convection.

The multiphysics configuration generates significant upper-tropospheric fCRPS reductions in the tropical Western Hemisphere and over the so-called Maritime Continent (Figs. 7a,b). Because the latter has important implications for long-range predictions (Dias and Kiladis 2019), the source of this sensitivity will be investigated in section 3b(1). The stochastic schemes make minimal contributions to upper-tropospheric forecast skill (Fig. 7), a disappointing result given their large impacts on spread (Fig. 2). Because the RHS of Eq. (3) represents the difference between MAE and inter-member departures, the SPPT- and SPP-induced spread must be locally offset by increased error. A preliminary analysis suggests that the mixing length error model (ml_emod in Part I) negatively impacts MAE aloft over Eurasia (not shown), implying that adjustments to individual elements may improve the performance of the SPP-based ensemble.

Fig. 7.
Fig. 7.

Contributions of the (a)–(d) multiphysics, (e)–(h) SPPT, and (i)–(l) SPP schemes to the upper-air fCRPS. Plotting conventions follow those used in Fig. 2. The North Atlantic region used in Figs. 11 and 12 is outlined in black on 500-hPa heights and 850-hPa temperatures. Also outlined in black for 500-hPa heights is the Maritime Continent region used later in Fig. 10 and Table 4.

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

The stochastic schemes’ contributions to forecast skill are structurally similar in the lower troposphere (Figs. 7g,k,h,l), with significant improvements primarily restricted to the stratocumulus regions (Wood 2012). Most striking at these levels, however, is the large fCRPS deterioration generated by the multiphysics configuration along the oceanic storm tracks (Fig. 7c). Unlike the multiphysics spread increase (Fig. 2), temperature and moisture sensitivities are decoupled: temperature forecasts are negatively affected in the higher latitudes (Fig. 7c), while moisture degradations are restricted to the tropics (Fig. 7d). The source of forecast skill deteriorations in the storm tracks will be investigated in section 3b(2).

Table 4

Vertically integrated apparent heating rate (Q1) over the Maritime Continent (Fig. 7), averaged over pentad 1 for the ensembles indicated in the first column. The integral of Q1 (Yanai et al. 1973) is converted to units equivalent to rainfall and evaporation through [1/(ρwL)]stQ1(dp/g), where ρw is the density of water, L is the latent heat of vaporization, p is pressure, g is gravitational acceleration, and the “s” and “t” limits refer to the surface and model top, respectively [similar to Ling and Zhang (2013) but including diabatic processes in the full column]. All differences w.r.t. the control ensembles (Table 3) are significant at the 95% level as based on a 1000-member bootstrap across cases. Naming conventions for subensembles follow the description for Fig. 10.

Table 4

The sensitivities observed at the surface (Fig. 8) are consistent with those noted at the top of the boundary layer. The multiphysics configuration again suffers from significant skill deteriorations in the storm tracks and the Arctic, while the stochastic schemes contribute significantly to continental forecast skill, particularly over elevated terrain. The spatial pattern of sea level pressure fCRPS reductions suggests a connection to the semidiurnal tide (Dai and Wang 1999), possibly a response to perturbation-induced stabilization of this oscillatory system (Berner et al. 2018). The sensitivity of this process to perturbation time scales (τ in Table 2) will be assessed in more detail in a future investigation.

Fig. 8.
Fig. 8.

Contribution of the (a),(b) multiphysics; (c),(d) SPPT; and (e),(f) SPP schemes to the near-surface fCRPS. Plotting conventions follow those used in Fig. 3. The Maritime Continent region used later in Fig. 10 and Table 4 is outlined in black on screen-level temperatures.

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

A flow-dependent evaluation of the spread–error relationship can be made through spread–reliability diagrams (Leutbecher et al. 2007). The interpretation of the results is complicated here by the fact that different controls are required for the multiphysics and stochastic-perturbation experiments (Table 3). Nevertheless, some key findings for 850-hPa temperatures emerge particularly at early lead times when the effects of the differing model uncertainty representations are most clearly distinguishable (Fig. 9).

Fig. 9.
Fig. 9.

Spread–reliability diagrams (Leutbecher et al. 2007) for 850-hPa temperature in 72-h forecasts over the (a),(c) northern midlatitudes and (b),(d) tropics for (left) the multiphysics configuration and (right) the stochastic schemes. Naming and color coding follow Table 3, with integrations grouped by their use of the updated physics suite so that each experiment is directly compared with the appropriate control ensemble. The climatological standard deviations used to normalize the spread (abscissa) and error (ordinate) values are computed for the 1989–2016 period from the ERA-Interim reanalysis (Dee et al. 2011). The means of the 20 percentile-based spread bins are plotted with circles, and a color-coded square is placed at the overall mean of each distribution.

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

In the northern (winter) midlatitudes, the multiphysics configuration increases error without generating supplemental spread as evidenced by translation parallel to the ideal 1:1 reference (Fig. 9a). Conversely, the stochastic schemes both induce shifts toward larger spread values and the diagonal without error increases (Fig. 9c). A flattening of the slope of the associated distributions, however, indicates sensitivity4 reduction and suggests that the stochastic schemes fail to represent the full range of error sources under specific conditions. For the SPP scheme in particular, this problem should be addressed through the development of additional SPP elements that accurately sample the relevant uncertainties.

Impacts are reversed in the tropics, where it is the multiphysics configuration that flattens the distribution in the spread–reliability diagram (Fig. 9b). This is consistent with reduced sensitivity arising from flow-independent differences between members [section 3a(1)]. The SPP scheme yields the best spread–reliability relationship in the tropics, with a distribution that lies close to the diagonal (Fig. 9d).

1) Case study: Heating over the Maritime Continent

The multiphysics configuration yields significant improvements in 500-hPa height forecasts over the Maritime Continent that are not afforded by either of the stochastic schemes (Figs. 7b,f,j). Although the dynamical relevance of tropical heights is limited, similar improvements in screen-level temperatures (Figs. 8a,c,e) suggest that this sensitivity merits further investigation.

The multiphysics configuration does not increase spread over the Maritime Continent (Fig. 2b), an indication that MAE reductions are responsible for fCRPS decreases [Eq. (3)]. Both 500-hPa heights and screen-level temperatures over the region are essentially bias-free in the multiphysics-based ensemble mean, in contrast to cold, low-height errors that prevail in the SPPT- and SPP-based systems (Fig. 10).

Fig. 10.
Fig. 10.

Cumulative distribution functions for pentad-1 (a) 500-hPa geopotential heights and (b) screen-level temperatures averaged over the Maritime Continent region shown in Fig. 7 in the ensembles indicated on the legend (Table 3). The multiphysics-based results are divided into subsets that are based on member configurations for the saltwater correction (“Salt = T” if the correction is active and “Salt = F” if it is not) and the value of the Prandtl number, with gray shading used to represent the subensembles as shown in the legend in the lower-right corner of the plot. The median for each ensemble and subensemble is shown at the bottom of the panel, with shading to indicate the 95% confidence interval based on a 1000-member bootstrap. The medians are plotted on separate rows for readability, with dashed lines connecting the full ensemble medians to their values within the distributions (intersection with the 0.5 level of the cumulative relevant distribution function).

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

Decomposition of the multiphysics-based results into subensembles distinguished by parameters that directly affect surface fluxes shows that member-specific climates combine to reduce the bias of the ensemble mean (Fig. 10 and Table 1). The reduced turbulent Prandtl number (0.85) increases scalar fluxes, leading to a clear separation in screen-level temperatures between the subensembles (Fig. 10b). Ignoring salinity inflates latent heat fluxes, whose impact is felt aloft through the promotion of convective heating (Fig. 10a). As shown in Table 4, changes to the apparent heat source [denoted Q1 by Yanai et al. (1973)] across the subensembles align well with 500-hPa height biases. The observed sensitivity is therefore a surface flux-modulated hydrostatic response to column-integrated condensation heating in this convectively active region (Ling and Zhang 2013). Although effective in minimizing ensemble-mean errors, the differing member climatologies once again poorly represent flow-dependent uncertainty.

The SPPT- and SPP-based ensembles account for salinity and Prandtl number effects to improve modeled oceanic fluxes and minimize precipitation overestimates (McTaggart-Cowan et al. 2019); however, decreased diabatic heating leads to cold biases over the Maritime Continent that are detrimental to MSE component of the fCRPS (Fig. 10). Future development efforts should prioritize bias reductions in this region because of its importance for long-range predictability (Vitart 2017; Dias and Kiladis 2019).

2) Case study: Forecast skill in the storm tracks

The behavior of the three model error representations is noticeably different in the oceanic storm tracks of the Northern Hemisphere (Figs. 7 and 8). The stochastic schemes have little impact on the fCRPS in these regions, while the multiphysics configuration significantly degrades lower-tropospheric temperature forecasts. The presence of this deterioration in an operational configuration is surprising; however, problems specific to oceanic domains may remain undetected because the GEPS is usually evaluated against radiosonde observations that do not directly sample these regions.

The forecast skill degradation in the multiphysics configuration is the result of increased MAE [Eq. (3)] rather than spread reduction (Fig. 2c). Members’ 850-hPa temperature errors depend strongly on the amplitude of a warm bias related to the thermal roughness length over water (Fig. 11b and Table 1). This quantity modulates turbulent surface heat fluxes, with the momentum-based formulation leading to much larger transfers during cold outbreaks triggered by midlatitude cyclones in the region (Tilinina et al. 2018). The excessive warming (Fig. 11b) leads to an undesirable upward shift parallel to the diagonal in spread–reliability diagram for the region (Fig. 12b). Although this heating also induces weak 500-hPa height biases, it does not increase MAE, as evidenced by a rightward shift with respect to the control ensemble in Fig. 11a. This leads to a height distribution that is practically superposed on the ideal 1:1 reference in the spread–reliability diagram (Fig. 12a).

Fig. 11.
Fig. 11.

Relationship between pentad-1 bias (abscissa) and MAE (ordinate) over the North Atlantic region (Fig. 7) for (a),(c),(e) 500-hPa height and (b),(d),(f) 850-hPa temperature in ensembles using the (left) multiphysics configuration; (center) SPPT; and (right) SPP schemes. Kernel density estimates are plotted for each experiment in contours that are color coded according to Table 3, normalized by sample size and plotted at unit density intervals from 1 to 5. Also shown are the distributions of the deterministic control member (thin black contours) and of the control ensemble (color shading) in which the relevant model error representation is absent (Table 3). The multiphysics ensemble is decomposed into subensembles (solid and dashed contours) distinguished by their treatment of the roughness length for scalars, which directly affects heat fluxes over water (Table 1).

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

Fig. 12.
Fig. 12.

Spread–reliability diagrams for 72-h forecasts of (a),(c) 500-hPa height and (b),(d) 850-hPa temperature over the North Atlantic region (Fig. 7) in ensembles using (left) the multiphysics configuration and (right) the stochastic schemes. Plotting follows the conventions described in Fig. 9.

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

The stochastic schemes yield 850-hPa temperature biases that are much closer to those of the control ensemble (Figs. 11d and 11f), with slight perturbation-induced cooling as noted in section 3a(1). The broadened temperature distribution in the SPP-based ensemble is indicative of larger spread; however, the SPPT scheme produces a solution that lies closest to the diagonal in the spread–reliability diagram (Fig. 12d). The 500-hPa height MAE values in the stochastically perturbed ensembles are systematically increased (vertically shifted from the control ensemble in Figs. 11c and 11e), implying that the schemes trigger significant perturbation growth in this dynamically active region. Both lead to overdispersion of the 500-hPa heights (Fig. 12c) and flatten the distribution away from the diagonal on the spread–reliability diagram.

These results are consistent with those obtained for the Pacific basin (not shown) and demonstrate that although the multiphysics configuration suffers from bias-induced skill degradation in the storm tracks, the spread introduced by the stochastic schemes is excessive. A modest reduction of selected perturbation amplitudes should reduce MAE values toward those of the control ensemble (Fig. 11e) while simultaneously reducing overdispersion (Figs. 12c,d). The potential benefit of such adjustments to the SPP scheme will be investigated as part of ongoing system development.

4. SPP in the Canadian Global Ensemble Prediction System

A major upgrade to the operational GEPS took place in November 2021, a stepwise reconstruction of which is presented in this section. Building on the isolated contributions assessed above, the impact of adopting the SPP-based configuration is documented in section 4a. This analysis is extended to include the effects of the updated model physics suite in section 4b. The performance of the full set of changes to the system, including those made to the assimilation component, is evaluated in section 4c. The experiments used for these discussions are described in Table 5.

Table 5

Description of GEPS configurations used in section 4, following the layout of Table 3. Unlike in Table 3, however, the coefficient for the kinetic energy backscatter scheme is changed between the systems in the last row as described in the section 4c.

Table 5

a. Step 1: Updating the model error representation

The combination of SPPT and multiphysics schemes (“SPPT+multiphysics”) used in the operational GEPS can only be replaced by SPP if the latter preserves or improves guidance quality. The internal consistency of parameterizations and conservation properties of the SPP scheme (Part I) will only tilt the balance toward its adoption under such conditions.

The SPP scheme’s overall contribution to ensemble spread is generally equivalent to that of the SPPT+multiphysics approach (Figs. 13a–c), in which complementary regional maxima lead to global spread distributions (Figs. 2a–h). Upper-tropospheric spread in the extratropics at short lead times is smaller in the SPP-based ensemble, consistent with the SPPT scheme’s enhanced activity aloft (Figs. 2e,f,i,j). In the tropics, the multiphysics configuration contributes to enhanced variability at the top of the boundary layer that is not matched in the SPP-based ensemble (Fig. 13b as seen in Figs. 2c,d,k,l). However, this spread is the result of multiple preferred states within the ensemble and does not accurately represent the flow-dependent uncertainty [section 3a(1)].

Fig. 13.
Fig. 13.

Summary statistics of the “step 1” transition from the combined SPPT+multiphysics approach to the SPP scheme, shown as the change in Cf to (a)–(c) ensemble spread and (d)–(f) fCRPS. To eliminate the effects of the differing controls, this sensitivity is diagnosed using Cf differences rather than the contributions themselves (Table 5); however, the plotting conventions follow those employed for Fig. 1. Annotations with arrows indicate the interpretation of sign of Cf as a reference.

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

The SPP scheme’s contribution to improved ensemble forecast skill is compared with that of the SPPT+multiphysics configuration in Figs. 13d–f. In the midlatitudes, the SPP scheme’s beneficial impact on the fCRPS at lower levels is highlighted by its contrast to the multiphysics-induced degradations [section 3b(2)]. At upper levels, however, the SPPT+multiphysics approach yields fCRPS reductions in the winter midlatitudes that are not matched by the SPP scheme at short lead times.

Although all model error representations significantly improve ensemble forecast skill in the tropics (Figs. 6b,e,h), the SPP scheme does not reduce the fCRPS for 500-hPa heights or screen-level temperatures as effectively as the combined approach (Fig. 13e). However, the relative improvements originating in the multiphysics component are the result of member-specific bias compensation rather than improved depictions of random errors in the system [section 3b(1)]. Improving tropical fCRPS in an SPP-based ensemble will require development efforts specifically aimed at bias reduction in the deterministic context. In the meantime, these results show that the SPP scheme represents a viable alternative to the model uncertainty estimates currently used in the GEPS.

b. Step 2: Adding the updated physical parameterization suite

A complicating factor throughout this study has been the absence of an effective multiphysics configuration within the updated suite of physical parameterizations. Without a strong multiphysics influence, the SPPT scheme is unable to reduce fCRPS as effectively as SPP (Fig. 6). This makes the adoption of the SPP scheme a prerequisite for the model physics upgrade in the GEPS.

The updated parameterizations have minimal impact on ensemble spread in the extratropics; however, SPP-induced screen-level temperature spread gains at lower latitudes are lost (cf. Figs. 13b and 14b). This sensitivity arises primarily from two changes within the suite: the introduction of a minimum Obukov length to prevent surface-layer decoupling, and a deepening of the deep convective downdraft detrainment depth (McTaggart-Cowan et al. 2019). Both of these modifications reduce the prevalence of highly localized and intermittent processes, thereby restricting inter-member differences (not shown). Future physics development efforts should assess the impacts of changes on higher-order moments of the system, an expansion of the current focus on the deterministic and ensemble-mean forecast skill.

Fig. 14.
Fig. 14.

Summary statistics following the “step 2” addition of the updated physics suite. As shown in Table 5, this sensitivity is computed using the SPP-based ensemble as “experiment” and the combined SPPT+multiphysics ensemble as “control” in Eq. (1). Plotting conventions follow those employed for Fig. 1.

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

The improved performance of the updated physics suite (McTaggart-Cowan et al. 2019) translates into significant fCRPS improvements (Figs. 14d–f). The only field that remains degraded in comparison with the SPPT+multiphysics ensemble is the pentad-3 tropical 500-hPa height (Fig. 14e). Overall, the SPP-based model configuration with updated physics significantly improves the quality of GEPS forecasts.

c. Step 3: Adding non-model GEPS updates

The remaining major components of the 2021 GEPS upgrade are external to the forecast model and include the replacement of the ensemble Kalman filter with a local ensemble transform Kalman filter (Buehner 2020) and modifications to the analysis error estimate. One exception is a set of adjustments made to increase the activity of the stochastic kinetic energy backscatter scheme (Charron et al. 2010): increasing the backscattering coefficient (from 0.4 to 1) and including tendencies from convective momentum transport (Shutts 2015). Both of these adjustments only induce small spread increases in the GEPS (not shown).

Ensemble spread is significantly reduced in the updated GEPS for almost all variables and lead times (Figs. 15a–c). This sensitivity arises primarily from a reduction in initialization diversity, which itself depends heavily on additive inflation (Houtekamer et al. 2009) determined by a rescaling of NMC background error covariances (Parrish and Derber 1992). The variance scaling factor was reduced from 0.66 to 0.43 (a standard deviation reduction of ∼20%) for GEPS initializations as part of the 2021 upgrade.

Fig. 15.
Fig. 15.

Results of the “step 3” full GEPS upgrade, including all external changes to the system. As shown in Table 5, this sensitivity is computed by using the updated system as experiment and the current operational system as control in Eq. (1). Blue shading is indicative of forecast skill improvement. Plotting conventions follow those employed for Fig. 1.

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

The reason that such an ensemble spread reduction is permissible is evident in Figs. 15d–f. The fCRPS is significantly reduced for the majority of variables and lead times, indicative of MAE reductions whose impacts on forecast skill counteract the spread decrease [Eq. (3)]. These improvements are consistent with those expected from the adoption of SPP and updated model physics (Fig. 14), with additional gains achieved in tropical predictions (cf. Figs. 14e and 15e). The only significant fCRPS deterioration remains the long-range tropical 500-hPa height forecasts (pentad 3 in Fig. 15e), a direct result of the elimination of multiphysics bias compensations [section 3b(1)].

The sensitivities highlighted by the fCRPS summary statistics are reflected in global spread–error evolution and spread–reliability diagrams (Figs. 16 and 17). Despite the decreased amplitude of initial-condition perturbations, spread growth keeps pace with unbiased ensemble-mean RMSE [RMSEu; Eq. (5) of Part I]5 aloft to yield a well-balanced system on the global scale (Fig. 16). Rapid spread growth for 850-hPa temperature in the updated GEPS largely compensates for underdispersion in the initializing analyses (Fig. 16c). From the spread–reliability perspective, the distribution of updated GEPS forecasts shifts along the diagonal toward the origin for all fields, an indication that spread decreases correlate with error reductions (Fig. 17). The modest flattening of the 500-hPa height distribution for large-spread events (Fig. 17b) is a tropically dominated signal (not shown), an indication that locally matching low-latitude spread and error should be a focus of future model development. Overall, the improvements in GEPS forecast skill shown in Figs. 1517 are the largest seen over the last decade.

Fig. 16.
Fig. 16.

Global ensemble spread (dashed) and ensemble mean unbiased RMSEu (solid) growth in the current operational GEPS (blue) and updated GEPS configuration (red) for (a) 250-hPa zonal winds, (b) 500-hPa height, and (c) 850-hPa temperatures. The vertical dashed gray lines indicate the 72-h forecast time used for the spread–reliability diagrams in Fig. 17, below.

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

Fig. 17.
Fig. 17.

Global spread–reliability diagrams for 72-h forecasts from the current operational GEPS (blue) and updated GEPS configuration (red) for (a) 250-hPa zonal winds, (b) 500-hPa heights, and (c) 850-hPa temperatures. Plotting follows the conventions described in Fig. 9.

Citation: Monthly Weather Review 150, 11; 10.1175/MWR-D-21-0316.1

5. Discussion

Accurate representation of model uncertainty is an important ingredient in the design of a reliable ensemble forecasting system. Most existing operational ensembles employ either the SPPT scheme or multiphysics configurations because they are highly effective at generating a large diversity of member solutions. However, these techniques suffer from conceptual limitations in the form of potential internal inconsistencies or the development of member-specific climates. The SPP scheme introduced in Part I of this study is an attempt to represent the broad range of potential error sources within the model in a self-consistent, flow-dependent way.

Forecasts produced by an SPP-based system are of comparable quality to those generated using existing representations of model uncertainty. The magnitude of ensemble spread induced by SPP is similar to that rendered by the SPPT scheme in the midlatitudes, and trails only the multiphysics configuration in the tropics. Spread in the latter, however, is inflated by systematic differences between ensemble members that lead to undesirable multimodal distributions. The multiphysics-based ensemble also generates large errors in the midlatitude storm tracks due to suboptimal physics configurations.

The use of combined SPPT+multiphysics in the operational GEPS yields additional benefits in the form of a global distribution of spread and improved tropical biases; however, the SPP scheme in isolation is found to yield forecasts of similar overall quality. When combined with an updated physical parameterization suite, the SPP-based ensemble significantly outperforms the existing system. As a result, the SPP technique was adopted in the operational GEPS in autumn of 2021.

The goal of model uncertainty representations is to depict the impact of random model errors on ensemble forecasts. As a result, the projection of biases onto ensemble statistics complicates the interpretation of results by favoring some configurations simply because of changes to the mean state (Hamill and Colucci 1997; Wang et al. 2018). The GEPS transition to SPP implies that biases in the ensemble mean will more closely resemble those of the control member and that they can no longer be reduced by changing the prevalence of specific multiphysics components. Future model development goals should therefore emphasize the reduction of systematic errors, possibly through a shift in focus from the bias-insensitive error standard deviation to metrics such as RMSE. Such an effort will benefit all GEM-based prediction systems and will reduce the SPP-induced spread needed to achieve a well-balanced GEPS.

The use of a broad range of system evaluation techniques can also lead to conflicting signals about forecast behavior. Although all of the metrics typically used to assess medium-range forecast quality suggest that the SPP-based GEPS is superior to the configuration that it replaces, an evaluation of the system’s ability to predict the Madden–Julian oscillation (Madden and Julian 1971) on seasonal time scales points to significant deterioration. Signs of similar degradation in an SPP-based ensemble were noted by Leutbecher et al. (2017). Preliminary investigations suggest that the GEPS problems may be related to biases in tropical outgoing longwave radiation, an important component of the index used to identify this convective feature that is highly sensitive to model uncertainty representations (Weisheimer et al. 2014; Li et al. 2019). Because the GEPS serves as the basis for monthly predictions issued by the CMC, additional efforts will be required to identify the source of this deterioration and to adjust the SPP scheme to improve long-range guidance.

Although the global ensemble was the focus of this investigation, the SPP technique was also implemented in the 2021 upgrade of the operational Canadian Regional Ensemble Prediction System. Despite the fact that this system had already adopted the updated physics suite in a 2019 upgrade, SPP led to significant improvements in forecasts on the continental scale and eliminated a long-standing near-surface wind error in the system (Patoine and Separovic 2021). Because forecast products from both systems are shared with NOAA to generate multimodel ensemble guidance, the improvements documented here promise to have positive impacts on probabilistic forecasts across North America.

The SPP scheme is a viable alternative to the SPPT- and multiphysics-based approaches that currently serve as the de facto standards for model error representation in operational ensemble forecasting systems around the world. The 2021 operationalization of the SPP-based GEPS represents an important proof of concept for the SPP method, with further improvements expected as the technique is refined in the future.

1

This configuration is referred to hereinafter as the “current operational” GEPS (version 6.1.0) despite its replacement by the system described here in December 2021 (version 7.0.0).

2

The configuration of the stochastic kinetic energy backscatter scheme does not change between each experiment and its respective control unless explicitly noted (section 4c).

3

All configurations use identical member-specific initial conditions and seeding of the stochastic kinetic energy backscatter scheme.

4

The term “sensitivity” is used in the context of spread–reliability diagrams to refer to the ensemble’s ability to distinguish between high- and low-predictability events through changes in ensemble spread.

5

The term “unbiased” in this context refers to the statistical manipulation to account for limited ensemble size, rather than any sort of postprocessing of the ensemble.

Acknowledgments.

The authors thank Dr. Hannah Christensen, two anonymous reviewers, and the editor (Dr. Tomasso Benacchio) for insightful and constructive suggestions that helped to prepare this study for publication.

Data availability statement.

Given the large volume of model outputs analyzed in this study, transfer to an independent repository is impractical; however, all data used in this study will be made freely available upon request. The full ERA Interim reanalysis dataset is archived by Copernicus Climate Services (https://climate.copernicus.eu/). The GEM model, including the implementation of SPP described here, is available online (https://github.com/ECCC-ASTD-MRD/gem).

REFERENCES

  • Ayar, P. V., M. Vrac, and A. Mailhot, 2021: Ensemble bias correction of climate simulations: Preserving internal variability. Sci. Rep., 11, 3098, https://doi.org/10.1038/s41598-021-82715-1.

    • Search Google Scholar
    • Export Citation
  • Baumgart, M., P. Ghinassi, V. Wirth, T. Selz, G. C. Craig, and M. Riemer, 2019: Quantitative view on the processes governing the upscale error growth up to the planetary scale using a stochastic convection scheme. Mon. Wea. Rev., 147, 17131731, https://doi.org/10.1175/MWR-D-18-0292.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bechtold, P., N. Semane, P. Lopez, J. P. Charboureau, A. Beljaars, and N. Bormann, 2014: Representing equilibrium and nonequilibrium convection in large-scale models. J. Atmos. Sci., 71, 734753, https://doi.org/10.1175/JAS-D-13-0163.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., J. Mailhot, C. Girard, and P. Vaillancourt, 2005: Boundary layer and shallow cumulus clouds in a medium-range forecast of a large-scale weather system. Mon. Wea. Rev., 133, 19381960, https://doi.org/10.1175/MWR2958.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berner, J., S.-Y. Ha, J. P. Hacker, A. Fournier, and C. Snyder, 2011: Model uncertainty in a mesoscale ensemble prediction system: Stochastic versus multiphysics representations. Mon. Wea. Rev., 139, 19721995, https://doi.org/10.1175/2010MWR3595.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berner, J., K. R. Fossell, S.-Y. Ha, J. P. Hacker, and C. Snyder, 2015: Increasing the skill of probabilistic forecasts: Understanding performance improvements from model-error representations. Mon. Wea. Rev., 143, 12951320, https://doi.org/10.1175/MWR-D-14-00091.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berner, J., and Coauthors, 2017: Stochastic parameterization: Toward a new view of weather and climate models. Bull. Amer. Meteor. Soc., 98, 565588, https://doi.org/10.1175/BAMS-D-15-00268.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berner, J., P. D. Sardeshmukh, and H. M. Christensen, 2018: On the dynamical mechanism governing El Ninõ–Southern Oscillation irregularity. J. Climate, 31, 84018419, https://doi.org/10.1175/JCLI-D-18-0243.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bertossa, C., P. Hitchcock, A. DeGaetano, R. Plougonven, 2021: Bimodality in ensemble forecasts of 2-meter temperature: Identification. Wea. Climate Dyn., 2, 12901224, https://doi.org/10.5194/wcd-2-1209-2021.

    • Search Google Scholar
    • Export Citation
  • Blackadar, A. K., 1962: The vertical distribution of wind and turbulent exchange in a neutral atmosphere. J. Geophys. Res., 67, 30953102, https://doi.org/10.1029/JZ067i008p03095.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bocquet, M., C. A. Pires, and L. Wu, 2010: Beyond Gaussian statistical modeling in geophysical data assimilation. Mon. Wea. Rev., 138, 29973023, https://doi.org/10.1175/2010MWR3164.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bougeault, P., and P. Lacarrère, 1989: Parameterization of orography-induced turbulence in a mesobeta-scale model. Mon. Wea. Rev., 117, 18721890, https://doi.org/10.1175/1520-0493(1989)117<1872:POOITI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buehner, M., 2020: Local ensemble transform Kalman filter with cross validation. Mon. Wea. Rev., 148, 22652282, https://doi.org/10.1175/MWR-D-19-0402.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buehner, M., and Coauthors, 2015: Implementation of deterministic weather forecasting systems based on ensemble–variational data assimilation at Environment Canada. Part I: The global system. Mon. Wea. Rev., 143, 25322559, https://doi.org/10.1175/MWR-D-14-00354.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buizza, R., M. Miller, and T. N. Palmer, 1999: Stochastic representation of model uncertainties in the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 125, 28872908, https://doi.org/10.1002/qj.49712556006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buizza, R., P. L. Houtekamer, G. Pierre, Z. Toth, Y. Zhu, and M. Wei, 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Mon. Wea. Rev., 133, 10761097, https://doi.org/10.1175/MWR2905.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Charnock, H., 1955: Wind stress on a water surface. Quart. J. Roy. Meteor. Soc., 81, 639640, https://doi.org/10.1002/qj.49708135027.

  • Charron, M., G. Pellerin, L. Spacek, P. L. Houtekamer, N. Gagnon, H. L. Mitchell, and L. Michelin, 2010: Toward random sampling of model error in the Canadian ensemble prediction system. Mon. Wea. Rev., 138, 18771901, https://doi.org/10.1175/2009MWR3187.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, J., J. Wang, J. Du, Y. Xia, F. Chen, and L. Hongqi, 2020: Forecast bias correction through model integration: A dynamical wholesale approach. Quart. J. Roy. Meteor. Soc., 146, 11491168, https://doi.org/10.1002/qj.3730.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Christensen, H. M., 2020: Constraining stochastic parameterization schemes using high-resolution simulations. Quart. J. Roy. Meteor. Soc., 146, 938962, https://doi.org/10.1002/qj.3717.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Christensen, H. M., I. M. Moroz, and T. N. Palmer, 2015: Stochastic and perturbed parameter representations of model uncertainty in convection parameterization. J. Atmos. Sci., 72, 25252544, https://doi.org/10.1175/JAS-D-14-0250.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Christensen, H. M., S.-J. Lock, I. M. Moroz, and T. N. Palmer, 2017: Introducing independent patterns into the stochastically perturbed parameterization tendencies (SPPT) scheme. Quart. J. Roy. Meteor. Soc., 143, 21682181, https://doi.org/10.1002/qj.3075.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cui, B., Z. Toth, Y. Zhu, and D. Hou, 2012: Bias correction for global ensemble forecast. Wea. Forecasting, 27, 396410, https://doi.org/10.1175/WAF-D-11-00011.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dai, A., and J. Wang, 1999: Diurnal and semidiurnal tides in the global surface pressure fields. J. Atmos. Sci., 56, 38743891, https://doi.org/10.1175/1520-0469(1999)056<3874:DASTIG>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deacu, D., V. Fortin, E. Klyszejko, C. Spence, and P. D. Blanken, 2012: Predicting the net basin supply to the Great Lakes with a hydrometeorological model. J. Hydrometeor., 13, 17391759, https://doi.org/10.1175/JHM-D-11-0151.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dee, D. P., and Coauthors, 2011: The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Quart. J. Roy. Meteor. Soc., 137, 553597, https://doi.org/10.1002/qj.828.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dias, J., and G. N. Kiladis, 2019: The influence of tropical forecast errors on higher latitude predictions. Geophys. Res. Lett., 46, 44504459, https://doi.org/10.1029/2019GL082812.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ferro, C. A. T., D. S. Richardson, and A. P. Weigel, 2008: On the effect of ensemble size on the discrete and continuous ranked probability scores. Meteor. Appl., 15, 1924, https://doi.org/10.1002/met.45.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fricker, T. E., C. A. T. Ferro, and D. B. Stephenson, 2013: Three recommendations for evaluating climate predictions. Meteor. Appl., 20, 246255, https://doi.org/10.1002/met.1409.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Frogner, I. L., U. Andrae, P. Ollinaho, A. Hally, K. Hämäläinen, J. Kauhanen, K. L. Ivarsson, and D. Yazagi, 2022: Model uncertainty representation in a convection-permitting ensemble–SPP and SPPT in Harmon EPS. Mon. Wea. Rev., 150, 775795, https://doi.org/10.1175/MWR-D-21-0099.1.

    • Search Google Scholar
    • Export Citation
  • Gagnon, N., and Coauthors, 2013: Improvements to the Global Ensemble Prediction System (GEPS), version 2.0.3 to 3.0.0. Environment Canada Doc., 49 pp., https://collaboration.cmc.ec.gc.ca/cmc/cmoi/product_guide/docs/lib/op_systems/doc_opchanges/technote_geps300_20130213_e.pdf.

    • Search Google Scholar
    • Export Citation
  • Gross, M., S. Malardel, C. Jablonowski, and N. Wood, 2016: Bridging the (knowledge) gap between physics and dynamics. Bull. Amer. Meteor. Soc., 97, 137142, https://doi.org/10.1175/BAMS-D-15-00103.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., and S. J. Colucci, 1997: Verification of Eta-RSM short-range ensemble forecasts. Mon. Wea. Rev., 125, 13121327, https://doi.org/10.1175/1520-0493(1997)125<1312:VOERSR>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hou, D., Z. Toth, and Y. Zhu, 2006: A stochastic parameterization scheme within NCEP global ensemble forecast system. 18th Conf. on Probability and Statistics in the Atmospheric Sciences, Atlanta, GA, Amer. Meteor. Soc., 4.5, https://ams.confex.com/ams/pdfpapers/101401.pdf.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., 2011: The use of multiple parameterizations in ensembles. Proc. ECMWF Workshop on Representing Model Uncertainty and Error in Numerical Weather and Climate Prediction Models, Reading, United Kingdom, ECMWF, 163173.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., L. Lefaivre, J. Derome, H. Ritchie, and H. L. Mitchell, 1996: A system approach to ensemble prediction. Mon. Wea. Rev., 124, 12251242, https://doi.org/10.1175/1520-0493(1996)124<1225:ASSATE>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., H. L. Mitchell, and X. Deng, 2009: Model error representation in an operational ensemble Kalman filter. Mon. Wea. Rev., 137, 21262143, https://doi.org/10.1175/2008MWR2737.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jankov, I., and Coauthors, 2017: A performance comparison between multiphysics and stochastic approaches within a North American RAP ensemble. Mon. Wea. Rev., 145, 11611179, https://doi.org/10.1175/MWR-D-16-0160.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jankov, I., J. Beck, J. Wolff, M. Harrold, J. B. Olson, T. Smirnova, C. Alexander, and J. Berner, 2019: Stochastically perturbed parameterizations in an HRRR-based ensemble. Mon. Wea. Rev., 147, 153173, https://doi.org/10.1175/MWR-D-18-0092.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jeworrek, J., G. West, and R. Stull, 2021: WRF precipitation performance and predictability for systematically varied parameterizations over complex terrain. Wea. Forecasting, 36, 893913, https://doi.org/10.1175/WAF-D-20-0195.1.

    • Search Google Scholar
    • Export Citation
  • Kain, J. S., and J. M. Fritsch, 1993: Convective parameterization for mesoscale models: The Kain–Fritsch scheme. The Representation of Cumulus Convection in Numerical Models of the Atmosphere, Meteor. Monogr., No. 46, Amer. Meteor. Soc., 165170.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kalina, E. A., I. Jankov, T. Alcott, J. Olson, J. Beck, J. Berner, D. Dowell, and C. Alexander, 2021: A progress report on the development of the High-Resolution Rapid Refresh ensemble. Wea. Forecasting, 36, 791804, https://doi.org/10.1175/WAF-D-20-0098.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Krinner, G., V. Kharin, R. Roehrig, J. Scinocca, and F. Codron, 2020: Historically-based run-time bias corrections substantially improve model projections of 100 years of future climate change. Commun. Earth Environ., 1, 29, https://doi.org/10.1038/s43247-020-00035-0.

    • Search Google Scholar
    • Export Citation
  • Kuo, H. L., 1974: Further studies of the parameterization of the influence of cumulus convection on large scale flow. J. Atmos. Sci., 31, 12321240, https://doi.org/10.1175/1520-0469(1974)031<1232:FSOTPO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lang, S. T. K., S.-J. Lock, M. Leutbecher, P. Bechtold, and R. M. Forbes, 2021: Revision of the stochastically perturbed parameterizations model uncertainty scheme in the integrated forecasting system. Quart. J. Roy. Meteor. Soc., 147, 13641381, https://doi.org/10.1002/qj.3978.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lee, J. A., W. C. Kolczynski, T. C. McCandless, and S. E. Haupt, 2012: An objective methodology for configuring and down-selecting an NWP ensemble for low-level wind prediction. Mon. Wea. Rev., 140, 22702286, https://doi.org/10.1175/MWR-D-11-00065.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Leutbecher, M., 2019: Ensemble size: How suboptimal is less than infinity? Quart. J. Roy. Meteor. Soc., 145, 107128, https://doi.org/10.1002/qj.3387.

  • Leutbecher, M., R. Buizza, and L. Isaksen, 2007: Ensemble forecasting and flow-dependent estimates of initial uncertainty. ECMWF Workshop on Flow-Dependent Aspects of Data Assimilation, Reading, United Kingdom, ECMWF, https://www.ecmwf.int/sites/default/files/elibrary/2007/10731-ensemble-forecasting-and-flow-dependent-estimates-initial-uncertainty.pdf.

    • Search Google Scholar
    • Export Citation
  • Leutbecher, M., and Coauthors, 2017: Stochastic representations of model uncertainties at ECMWF: State of the art and future vision. Quart. J. Roy. Meteor. Soc., 143, 23152339, https://doi.org/10.1002/qj.3094.