A Performance Comparison between Multiphysics and Stochastic Approaches within a North American RAP Ensemble

Isidora Jankov Cooperative Institute for Research in the Atmosphere, Colorado State University, Fort Collins, Colorado
National Oceanic and Atmospheric Administration/Earth System Research Laboratory/Global Systems Division/Developmental Testbed Center, Boulder, Colorado

Search for other papers by Isidora Jankov in
Current site
Google Scholar
PubMed
Close
,
Judith Berner National Center for Atmospheric Research, Boulder, Colorado

Search for other papers by Judith Berner in
Current site
Google Scholar
PubMed
Close
,
Jeffrey Beck Cooperative Institute for Research in the Atmosphere, Colorado State University, Fort Collins, Colorado
National Oceanic and Atmospheric Administration/Earth System Research Laboratory/Global Systems Division/Developmental Testbed Center, Boulder, Colorado

Search for other papers by Jeffrey Beck in
Current site
Google Scholar
PubMed
Close
,
Hongli Jiang Cooperative Institute for Research in the Atmosphere, Colorado State University, Fort Collins, Colorado
National Oceanic and Atmospheric Administration/Earth System Research Laboratory/Global Systems Division/Developmental Testbed Center, Boulder, Colorado

Search for other papers by Hongli Jiang in
Current site
Google Scholar
PubMed
Close
,
Joseph B. Olson Cooperative Institute for Research in the Atmosphere, Colorado State University, Fort Collins, Colorado
National Oceanic and Atmospheric Administration/Earth System Research Laboratory, Boulder, Colorado

Search for other papers by Joseph B. Olson in
Current site
Google Scholar
PubMed
Close
,
Georg Grell National Oceanic and Atmospheric Administration/Earth System Research Laboratory, Boulder, Colorado

Search for other papers by Georg Grell in
Current site
Google Scholar
PubMed
Close
,
Tatiana G. Smirnova Cooperative Institute for Research in the Atmosphere, Colorado State University, Fort Collins, Colorado
National Oceanic and Atmospheric Administration/Earth System Research Laboratory, Boulder, Colorado

Search for other papers by Tatiana G. Smirnova in
Current site
Google Scholar
PubMed
Close
,
Stanley G. Benjamin National Oceanic and Atmospheric Administration/Earth System Research Laboratory, Boulder, Colorado

Search for other papers by Stanley G. Benjamin in
Current site
Google Scholar
PubMed
Close
, and
John M. Brown National Oceanic and Atmospheric Administration/Earth System Research Laboratory, Boulder, Colorado

Search for other papers by John M. Brown in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

A stochastic parameter perturbation (SPP) scheme consisting of spatially and temporally varying perturbations of uncertain parameters in the Grell–Freitas convective scheme and the Mellor–Yamada–Nakanishi–Niino planetary boundary scheme was developed within the Rapid Refresh ensemble system based on the Weather Research and Forecasting Model. Alone the stochastic parameter perturbations generate insufficient spread to be an alternative to the operational configuration that utilizes combinations of multiple parameterization schemes. However, when combined with other stochastic parameterization schemes, such as the stochastic kinetic energy backscatter (SKEB) scheme or the stochastic perturbation of physics tendencies (SPPT) scheme, the stochastic ensemble system has comparable forecast performance. An additional analysis quantifies the added value of combining SPP and SPPT over an ensemble that uses SPPT only, which is generally beneficial, especially for surface variables. The ensemble combining all three stochastic methods consistently produces the best spread–skill ratio and generally outperforms the multiphysics ensemble. The results of this study indicate that using a single-physics suite ensemble together with stochastic methods is an attractive alternative to multiphysics ensembles and should be considered in the design of future high-resolution regional and global ensembles.

Denotes content that is immediately available upon publication as open access.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author e-mail: Isidora Jankov, isidora.jankov@noaa.gov

Abstract

A stochastic parameter perturbation (SPP) scheme consisting of spatially and temporally varying perturbations of uncertain parameters in the Grell–Freitas convective scheme and the Mellor–Yamada–Nakanishi–Niino planetary boundary scheme was developed within the Rapid Refresh ensemble system based on the Weather Research and Forecasting Model. Alone the stochastic parameter perturbations generate insufficient spread to be an alternative to the operational configuration that utilizes combinations of multiple parameterization schemes. However, when combined with other stochastic parameterization schemes, such as the stochastic kinetic energy backscatter (SKEB) scheme or the stochastic perturbation of physics tendencies (SPPT) scheme, the stochastic ensemble system has comparable forecast performance. An additional analysis quantifies the added value of combining SPP and SPPT over an ensemble that uses SPPT only, which is generally beneficial, especially for surface variables. The ensemble combining all three stochastic methods consistently produces the best spread–skill ratio and generally outperforms the multiphysics ensemble. The results of this study indicate that using a single-physics suite ensemble together with stochastic methods is an attractive alternative to multiphysics ensembles and should be considered in the design of future high-resolution regional and global ensembles.

Denotes content that is immediately available upon publication as open access.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author e-mail: Isidora Jankov, isidora.jankov@noaa.gov
Save
  • Alexander, C., and Coauthors, 2016: Development of high-resolution Rapid Refresh (HRRR) ensemble data assimilation, forecasts and post processing. Proc. Seventh Ensemble User’s Workshop, College Park, MD, NCEP, 7.4. [Available online at http://www.dtcenter.org/events/workshops16/ensembles/docs/day2/7.4_-_Alexander_Curtis_-_Development_of_High-Resolution_Rapid_Refresh_(HRRR)_Ensemble_Data_Assimilation_Forecasts_and_Post_Processing.pdf.]

  • Benjamin, S. G., and Coauthors, 2016: A North American hourly assimilation and model forecast cycle: The Rapid Refresh. Mon. Wea. Rev., 144, 16691694, doi:10.1175/MWR-D-15-0242.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berner, J., G. Shutts, M. Leutbecher, and T. Palmer, 2009: A spectral stochastic kinetic energy backscatter scheme and its impact on flow-dependent predictability in the ECMWF ensemble prediction system. J. Atmos. Sci., 66, 603626, doi:10.1175/2008JAS2677.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berner, J., S.-Y. Ha, J. P. Hacker, A. Fournier, and C. Snyder, 2011: Model uncertainty in a mesoscale ensemble prediction system: Stochastic versus multiphysics representations. Mon. Wea. Rev., 139, 19721995, doi:10.1175/2010MWR3595.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berner, J., T. Jung, and T. N. Palmer, 2012: Systematic model error: The impact of increased horizontal resolution versus improved stochastic and deterministic parameterizations. J. Climate, 25, 49464962, doi:10.1175/JCLI-D-11-00297.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berner, J., K. R. Fossell, S.-Y. Ha, J. P. Hacker, and C. Snyder, 2015: Increasing the skill of probabilistic forecasts: Understanding performance improvements from model-error representations. Mon. Wea. Rev., 143, 12951320, doi:10.1175/MWR-D-14-00091.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Betts, A. K., 1986: A new convective adjustment scheme. Part I: Observational and theoretical basis. Quart. J. Roy. Meteor. Soc., 112, 677691, doi:10.1002/qj.49711247307.

    • Search Google Scholar
    • Export Citation
  • Bougeault, P., and P. Lacarrère, 1989: Parameterization of orography-induced turbulence in a mesobeta-scale model. Mon. Wea. Rev., 117, 18721890, doi:10.1175/1520-0493(1989)117<1872:POOITI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bouttier, F., B. Vié, O. Nuissier, and L. Raynaud, 2012: Impact of stochastic physics in a convection-permitting ensemble. Mon. Wea. Rev., 140, 37063721, doi:10.1175/MWR-D-12-00031.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bowler, N. E., A. Arribas, S. E. Beare, K. R. Mylne, and G. J. Shutts, 2009: The local ETKF and SKEB: Upgrades to the MOGREPS short-range ensemble prediction system. Quart. J. Roy. Meteor. Soc., 135, 767776, doi:10.1002/qj.394.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brier, G. W., 1950: Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78, 13, doi:10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buizza, R., M. Miller, and T. N. Palmer, 1999: Stochastic representation of model uncertainties in the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 125, 28872908, doi:10.1002/qj.49712556006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buizza, R., P. L. Houtekamer, Z. Toth, G. Pellerin, M. Wei, and Y. Zhu, 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Mon. Wea. Rev., 133, 10761097, doi:10.1175/MWR2905.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Candille, G., and O. Talagrand, 2005: Evaluation of probabilistic prediction systems for a scalar variable. Quart. J. Roy. Meteor. Soc., 131, 21312150, doi:10.1256/qj.04.71.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Candille, G., and O. Talagrand, 2008: Retracted and replaced: Impact of observational error on the validation of ensemble prediction systems. Quart. J. Roy. Meteor. Soc., 134, 509521, doi:10.1002/qj.221.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Charles, M. E., and B. A. Colle, 2009: Verification of extratropical cyclones within the NCEP operational models. Part II: The Short-Range Ensemble Forecast System. Wea. Forecasting, 24, 11911214, doi:10.1175/2009WAF2222170.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Christensen, H. M., I. M. Moroz, and T. N. Palmer, 2015: Stochastic and perturbed parameter representations of model uncertainty in convection parameterization. J. Atmos. Sci., 72, 25252544, doi:10.1175/JAS-D-14-0250.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Eckel, F. A., and C. F. Mass, 2005: Aspects of effective mesoscale, short-range ensemble forecasting. Wea. Forecasting, 20, 328350, doi:10.1175/WAF843.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Grell, G. A., and D. Dévényi, 2002: A generalized approach to parameterizing convection combining ensemble and data assimilation techniques. Geophys. Res. Lett., 29, 16931696, doi:10.1029/2002GL015311.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Grell, G. A., and S. R. Freitas, 2014: A scale and aerosol aware stochastic convective parameterization for weather and air quality modeling. Atmos. Chem. Phys., 14, 52335250, doi:10.5194/acp-14-5233-2014.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guan, H., B. Cui, and Y. Zhu, 2015: Improvement of statistical postprocessing using GEFS reforecast information. Wea. Forecasting, 30, 841854, doi:10.1175/WAF-D-14-00126.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hacker, J. P., C. Snyder, S.-Y. Ha, and M. Pocernich, 2011a: Linear and nonlinear response to parameter variations in a mesoscale model. Tellus, 63A, 429444, doi:10.1111/j.1600-0870.2010.00505.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hacker, J. P., and Coauthors, 2011b: The U.S. Air Force Weather Agency’s mesoscale ensemble: Scientific description and performance results. Tellus, 63A, 625641, doi:10.1111/j.1600-0870.2010.00497.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 1999: Hypothesis tests for evaluating numerical precipitation forecasts. Wea. Forecasting, 14, 155167, doi:10.1175/1520-0434(1999)014<0155:HTFENP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129, 550560, doi:10.1175/1520-0493(2001)129<0550:IORHFV>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Han, J., and H.-L. Pan, 2011: Revision of convection and vertical diffusion schemes in the NCEP Global Forecast System. Wea. Forecasting, 26, 520533, doi:10.1175/WAF-D-10-05038.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hersbach, H., 2000: Decomposition of the continuous ranked probability score for ensemble prediction systems. Wea. Forecasting, 15, 559570, doi:10.1175/1520-0434(2000)015<0559:DOTCRP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hong, S.-Y., Y. Noh, and J. Dudhia, 2006: A new vertical diffusion package with an explicit treatment of entrainment processes. Mon. Wea. Rev., 134, 23182341, doi:10.1175/MWR3199.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Janjić, Z. I., 1994: The step-mountain Eta coordinate model: Further developments of the convection, viscous sublayer, and turbulence closure schemes. Mon. Wea. Rev., 122, 927945, doi:10.1175/1520-0493(1994)122<0927:TSMECM>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Knutti, R., D. Masson, and A. Gettelman, 2013: Climate model genealogy: Generation CMIP5 and how we got there. Geophys. Res. Lett., 40, 11941199, doi:10.1002/grl.50256.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lin, Y., and K. E. Mitchell, 2005: The NCEP Stage II/IV hourly precipitation analyses: Development and applications. 19th Conf. on Hydrology, San Diego, CA, Amer. Meteor. Soc., 1.2. [Available online at https://ams.confex.com/ams/Annual2005/techprogram/paper_83847.htm.]

  • Mason, P., and D. Thomson, 1992: Stochastic backscatter in large-eddy simulations of boundary layers. J. Fluid Mech., 242, 5178, doi:10.1017/S0022112092002271.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Müller, W., C. Appenzeller, F. Doblas-Reyes, and M. Liniger, 2005: A debiased ranked probability skill score to evaluate probabilistic ensemble forecasts with small ensemble sizes. J. Climate, 18, 15131523, doi:10.1175/JCLI3361.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, J., D. Sexton, D. Barnett, G. Jones, M. Webb, M. Collins, and D. Stainforth, 2004: Quantification of modelling uncertainties in a large ensemble of climate change simulations. Nature, 430, 768772, doi:10.1038/nature02771.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nakanishi, M., and H. Niino, 2004: An improved Mellor–Yamada Level-3 model with condensation physics: Its design and verification. Bound.-Layer Meteor., 112, 131, doi:10.1023/B:BOUN.0000020164.04146.98.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nakanishi, M., and H. Niino, 2006: An improved Mellor–Yamada Level-3 model: Its numerical stability and application to a regional prediction of advection fog. Bound.-Layer Meteor., 119, 397407, doi:10.1007/s10546-005-9030-8.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., 2001: A nonlinear dynamical perspective on model error: A proposal for non-local stochastic-dynamic parameterization in weather and climate prediction. Quart. J. Roy. Meteor. Soc., 127, 279304, doi:10.1002/qj.49712757202.

    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., R. Buizza, F. Doblas-Reyes, T. Jung, M. Leutbecher, G. Shutts, M. Steinheimer, and A. Weisheimer, 2009: Stochastic parametrization and model uncertainty. ECMWF Tech. Memo. 598, 42 pp. [Available online at http://www.ecmwf.int/publications/.]

  • Reynolds, C. A., J. G. McLay, J. S. Goerss, E. A. Serra, D. Hodyss, and C. R. Sampson, 2011: Impact of resolution and design on the U.S. Navy global ensemble performance in the tropics. Mon. Wea. Rev., 139, 21452155, doi:10.1175/2011MWR3546.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Romine, G. S., C. S. Schwartz, J. Berner, K. R. Fossell, C. Snyder, J. L. Anderson, and M. L. Weisman, 2014: Representing forecast error in a convection-permitting ensemble system. Mon. Wea. Rev., 142, 45194541, doi:10.1175/MWR-D-14-00100.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sanchez, C., K. D. Williams, and M. Collins, 2015: Improved stochastic physics schemes for global weather and climate models. Quart. J. Roy. Meteor. Soc., 142, 147159. doi:10.1002/qj.2640.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schaefer, J. T., 1990: The critical success index as an indicator of warning skill. Wea. Forecasting, 5, 570575, doi:10.1175/1520-0434(1990)005<0570:TCSIAA>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shutts, G. J., 2005: A kinetic energy backscatter algorithm for use in ensemble prediction systems. Quart. J. Roy. Meteor. Soc., 131, 30793102, doi:10.1256/qj.04.106.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Skamarock, W. C., and Coauthors, 2008: A description of the Advanced Research WRF version 3. NCAR Tech. Note NCAR/TN-475+STR, 113 pp., doi:10.5065/D68S4MVH.

    • Crossref
    • Export Citation
  • Smirnova, T. G., J. M. Brown, S. G. Benjamin, and J. S. Kenyon, 2016: Modifications to the Rapid Update Cycle Land Surface Model (RUC LSM) available in the Weather Research and Forecast (WRF) Model. Mon. Wea. Rev., 144, 18511865, doi:10.1175/MWR-D-15-0198.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stensrud, D. J., H. E. Brooks, J. Du, M. S. Tracton, and E. Rogers, 1999: Using ensembles for short-range forecasting. Mon. Wea. Rev., 127, 433446, doi:10.1175/1520-0493(1999)127<0433:UEFSRF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74, 23172330, doi:10.1175/1520-0477(1993)074<2317:EFANTG>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Weaver, A., and P. Courtier, 2001: Correlation modelling on the sphere using a generalized diffusion equation. Quart. J. Roy. Meteor. Soc., 127, 18151846, doi:10.1002/qj.49712757518.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. International Geophysics Series, Vol. 100, Academic Press, 676 pp.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 4027 1827 93
PDF Downloads 1254 173 10