Editorial Type:
Article
Article Type:
Research Article

Mesoscale Model Evaluation Testbed (MMET): A Resource for Transitioning NWP Innovations from Research to Operations (R2O)

Jamie K. Wolff National Center for Atmospheric Research*/Research Applications Laboratory, and Developmental Testbed Center, Boulder, Colorado

Search for other papers by Jamie K. Wolff in
Current site
Google Scholar
PubMed Close
,
Michelle Harrold National Center for Atmospheric Research*/Research Applications Laboratory, and Developmental Testbed Center, Boulder, Colorado

Search for other papers by Michelle Harrold in
Current site
Google Scholar
PubMed Close
,
Tracy Hertneky National Center for Atmospheric Research*/Research Applications Laboratory, and Developmental Testbed Center, Boulder, Colorado

Search for other papers by Tracy Hertneky in
Current site
Google Scholar
PubMed Close
,
Eric Aligo National Centers for Environmental Prediction/Environmental Modeling Center, College Park, and IM Systems Group, Rockville, Maryland

Search for other papers by Eric Aligo in
Current site
Google Scholar
PubMed Close
,
Jacob R. Carley National Centers for Environmental Prediction/Environmental Modeling Center, College Park, and IM Systems Group, Rockville, Maryland

Search for other papers by Jacob R. Carley in
Current site
Google Scholar
PubMed Close
,
Brad Ferrier National Centers for Environmental Prediction/Environmental Modeling Center, College Park, and IM Systems Group, Rockville, Maryland

Search for other papers by Brad Ferrier in
Current site
Google Scholar
PubMed Close
,
Geoff DiMego National Centers for Environmental Prediction/Environmental Modeling Center, College Park, Maryland

Search for other papers by Geoff DiMego in
Current site
Google Scholar
PubMed Close
,
Louisa Nance National Center for Atmospheric Research*/Research Applications Laboratory, and Developmental Testbed Center, Boulder, Colorado

Search for other papers by Louisa Nance in
Current site
Google Scholar
PubMed Close
, and
Ying-Hwa Kuo National Center for Atmospheric Research*/Research Applications Laboratory, and Developmental Testbed Center, Boulder, Colorado

Search for other papers by Ying-Hwa Kuo in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Nov 2016
DOI:
https://doi.org/10.1175/BAMS-D-15-00001.1
Page(s):
2135–2147
Restricted access

Abstract

A wide range of numerical weather prediction (NWP) innovations are under development in the research community that have the potential to positively impact operational models. The Developmental Testbed Center (DTC) helps facilitate the transition of these innovations from research to operations (R2O). With the large number of innovations available in the research community, it is critical to clearly define a testing protocol to streamline the R2O process. The DTC has defined such a process that relies on shared responsibilities of the researchers, the DTC, and operational centers to test promising new NWP advancements. As part of the first stage of this process, the DTC instituted the mesoscale model evaluation testbed (MMET), which established a common testing framework to assist the research community in demonstrating the merits of developments. The ability to compare performance across innovations for critical cases provides a mechanism for selecting the most promising capabilities for further testing. If the researcher demonstrates improved results using MMET, then the innovation may be considered for the second stage of comprehensive testing and evaluation (T&E) prior to entering the final stage of preimplementation T&E.

MMET provides initialization and observation datasets for several case studies and multiday periods. In addition, the DTC provides baseline results for select operational configurations that use the Advanced Research version of Weather Research and Forecasting Model (ARW) or the National Oceanic and Atmospheric Administration (NOAA) Environmental Modeling System Nonhydrostatic Multiscale Model on the B grid (NEMS-NMMB). These baselines can be used for testing sensitivities to different model versions or configurations in order to improve forecast performance.

The National Center for Atmospheric Research is sponsored by the National Science Foundation.

CORRESPONDING AUTHOR: Jamie K. Wolff, NCAR/RAL, P.O. Box 3000, Boulder, CO 80307-3000, E-mail: jwolff@ucar.edu

Abstract

A wide range of numerical weather prediction (NWP) innovations are under development in the research community that have the potential to positively impact operational models. The Developmental Testbed Center (DTC) helps facilitate the transition of these innovations from research to operations (R2O). With the large number of innovations available in the research community, it is critical to clearly define a testing protocol to streamline the R2O process. The DTC has defined such a process that relies on shared responsibilities of the researchers, the DTC, and operational centers to test promising new NWP advancements. As part of the first stage of this process, the DTC instituted the mesoscale model evaluation testbed (MMET), which established a common testing framework to assist the research community in demonstrating the merits of developments. The ability to compare performance across innovations for critical cases provides a mechanism for selecting the most promising capabilities for further testing. If the researcher demonstrates improved results using MMET, then the innovation may be considered for the second stage of comprehensive testing and evaluation (T&E) prior to entering the final stage of preimplementation T&E.

MMET provides initialization and observation datasets for several case studies and multiday periods. In addition, the DTC provides baseline results for select operational configurations that use the Advanced Research version of Weather Research and Forecasting Model (ARW) or the National Oceanic and Atmospheric Administration (NOAA) Environmental Modeling System Nonhydrostatic Multiscale Model on the B grid (NEMS-NMMB). These baselines can be used for testing sensitivities to different model versions or configurations in order to improve forecast performance.

The National Center for Atmospheric Research is sponsored by the National Science Foundation.

CORRESPONDING AUTHOR: Jamie K. Wolff, NCAR/RAL, P.O. Box 3000, Boulder, CO 80307-3000, E-mail: jwolff@ucar.edu
Save
Cover Bulletin of the American Meteorological Society
Bulletin of the American Meteorological Society

  • Aligo, E., B. Ferrier, J. Carley, E. Rogers, M. Pyle, S. J. Weiss, and I. L. Jirak, 2014: Modified microphysics for use in high-resolution NAM forecasts. 27th Conf. on Severe Local Storms, Madison, WI, Amer. Meteor. Soc., 16A.1. [Available online at https://ams.confex.com/ams/27SLS/webprogram/Paper255732.html.]

  • Angevine, W. M., H. Jiang, and T. Mauritsen, 2010: Performance of an eddy diffusivity–mass flux scheme for shallow cumulus boundary layers. Mon. Wea. Rev., 138, 2895–2912, doi:10.1175/2010MWR3142.1.

    • Search Google Scholar
    • Export Citation

  • Benjamin, S. G., G. A. Grell, J. M. Brown, and T. G. Smirnova, 2004: Mesoscale weather prediction with the RUC hybrid isentropic–terrain-following coordinate model. Mon. Wea. Rev., 132, 473–494, doi:10.1175/1520-0493(2004)132<0473:MWPWTR>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Bernardet, L., and Coauthors, 2008: The Developmental Testbed Center and its Winter Forecasting Experiment. Bull. Amer. Meteor. Soc., 89, 611–627, doi:10.1175/BAMS-89-5-611.

    • Search Google Scholar
    • Export Citation

  • Bernardet, L., and Coauthors, 2014: Community support and transition of research to operations for the Hurricane Weather Research and Forecast (HWRF) model. Bull. Amer. Meteor. Soc., 96, 953–960, doi:10.1175/BAMS-D-13-00093.1.

    • Search Google Scholar
    • Export Citation

  • Dudhia, J., 1989: Numerical study of convection observed during the winter monsoon experiment using a mesoscale two-dimensional model. J. Atmos. Sci., 46, 3077–3107, doi:10.1175/1520-0469(1989)046<3077:NSOCOD>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Fowler, T. L., T. Jensen, E. I. Tollerud, J. Halley Gotway, P. Oldenburg, and R. Bullock, 2010: New Model Evaluation Tools (MET) software capabilities for QPF verification. Preprints, Third Int. Conf. on QPE/QPF and Hydrology, Nanjing, China, World Weather Research Programme, 189–193.

  • Fulker, D., S. Bates, and C. Jacobs, 1997: Unidata: A virtual community sharing resources via technological infrastructure. Bull. Amer. Meteor. Soc., 78, 457–468, doi:10.1175/1520-0477(1997)078<0457:UAVCSR>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Grell, G. A., and S. R. Freitas, 2014: A scale and aerosol aware stochastic convective parameterization for weather and air quality modeling. Atmos. Chem. Phys., 14, 5233–5250, doi:10.5194/acp-14-5233-2014.

    • Search Google Scholar
    • Export Citation

  • Guan, H., B. Cui, and Y. Zhu, 2015: Improvement of statistical postprocessing using GEFS reforecast information. Wea. Forecasting, 30, 841–854, doi:10.1175/WAF-D-14-00126.1.

    • Search Google Scholar
    • Export Citation

  • Hong, S.-Y., J. Dudhia, and S.-H. Chen, 2004: A revised approach to ice microphysical processes for the bulk parameterization of cloud and precipitation. Mon. Wea. Rev., 132, 103–120, doi:10.1175/1520-0493(2004)132<0103:ARATIM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Hong, S.-Y., Y. Noh, and J. Dudhia, 2006: A new vertical diffusion package with an explicit treatment of entrainment processes. Mon. Wea. Rev., 134, 2318–2341, doi:10.1175/MWR3199.1.

    • Search Google Scholar
    • Export Citation

  • Iacono, M. J., J. S. Delamere, E. J. Mlawer, M. W. Shephard, S. A. Clough, and W. D. Collins, 2008: Radiative forcing by long-lived greenhouse gases: Calculations with the AER radiative transfer models. J. Geophys. Res., 113, D13103, doi:10.1029/2008JD009944.

    • Search Google Scholar
    • Export Citation

  • Janjic, Z. I., 1994: The step–mountain eta coordinate model: Further developments of the convection, viscous sublayer, and turbulence closure schemes. Mon. Wea. Rev., 122, 927–945, doi:10.1175/1520-0493(1994)122<0927:TSMECM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Janjic, Z. I., 1996: The surface layer in the NCEP Eta model. Preprints, 11th Conf. on Numerical Weather Prediction, Norfolk, VA, Amer. Meteor. Soc., 354–355.

  • Jiménez, P. A., and J. Dudhia, 2012: Improving the representation of resolved and unresolved topographic effects on surface wind in the WRF Model. J. Appl. Meteor. Climatol., 51, 300–316, doi:10.1175/JAMC-D-11-084.1.

    • Search Google Scholar
    • Export Citation

  • Jiménez, P. A., and J. Dudhia, 2013: On the ability of the WRF Model to reproduce the surface wind direction over complex terrain. J. Appl. Meteor. Climatol., 52, 1610–1617, doi:10.1175/JAMC-D-12-0266.1.

    • Search Google Scholar
    • Export Citation

  • Jolliffe, I. T., and D. B. Stephenson, 2011: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. 2nd ed. Wiley, 292 pp.

  • Kain, J. S., 2004: The Kain–Fritsch convective parameterization: An update. J. Appl. Meteor., 43, 170–181, doi:10.1175/1520-0450(2004)043<0170:TKCPAU>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Mansell, E. R., C. L. Ziegler, and E. C. Bruning, 2010: Simulated electrification of a small thunderstorm with two-moment bulk microphysics. J. Atmos. Sci., 67, 171–194, doi:10.1175/2009JAS2965.1.

    • Search Google Scholar
    • Export Citation

  • Mlawer, E. J., S. J. Taubman, P. D. Brown, M. J. Iacono, and S. A. Clough, 1997: Radiative transfer for inhomogeneous atmospheres: RRTM, a validated correlated-k model for the longwave. J. Geophys. Res., 102, 16 663–16 682, doi:10.1029/97JD00237.

    • Search Google Scholar
    • Export Citation

  • Morrison, H., G. Thompson, and V. Tatarskii, 2009: Impact of cloud microphysics on the development of trailing stratiform precipitation in a simulated squall line: Comparison of one- and two-moment schemes. Mon. Wea. Rev., 137, 991–1007, doi:10.1175/2008MWR2556.1.

    • Search Google Scholar
    • Export Citation

  • Nakanishi, M., and H. Niino, 2006: An improved Mellor–Yamada level 3 model: Its numerical stability and application to a regional prediction of advecting fog. Bound.-Layer Meteor., 119, 397–407, doi:10.1007/s10546-005-9030-8.

    • Search Google Scholar
    • Export Citation

  • Niu, G.-Y., and Coauthors, 2011: The community Noah land surface model with multiparameterization options (Noah-MP): 1. Model description and evaluation with local-scale measurements. J. Geophys. Res., 116, D12109, doi:10.1029/2010JD015139.

    • Search Google Scholar
    • Export Citation

  • Paulson, C. A., 1970: The mathematical representation of wind speed and temperature profiles in the unstable atmospheric surface layer. J. Appl. Meteor., 9, 857–861, doi:10.1175/1520-0450(1970)009<0857:TMROWS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Pleim, J. E., 2007: A combined local and nonlocal closure model for the atmospheric boundary layer. Part I: Model description and testing. J. Appl. Meteor. Climatol., 46, 1383–1395, doi:10.1175/JAM2539.1.

    • Search Google Scholar
    • Export Citation

  • Roberts, N. M., and H. W. Lean, 2008: Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Mon. Wea. Rev., 136, 78–97, doi:10.1175/2007MWR2123.1.

    • Search Google Scholar
    • Export Citation

  • Sherretz, L. A., and D. W. Fulker, 1988: Unidata: Enabling universities to acquire and analyze scientific data. Bull. Amer. Meteor. Soc., 69, 373–376, doi:10.1175/1520-0477(1988)069<0373:UEUTAA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Tewari, M., and Coauthors, 2004: Implementation and verification of the unified Noah land surface model in the WRF model. 20th Conf. on Weather Analysis and Forecasting/16th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., 14.2a. [Available online at https://ams.confex.com/ams/84Annual/techprogram/paper_69061.htm.]

  • Thompson, G., and T. Eidhammer, 2014: A study of aerosol impacts on clouds and precipitation development in a large winter cyclone. J. Atmos. Sci., 71, 3636–3658, doi:10.1175/JAS-D-13-0305.1.

    • Search Google Scholar
    • Export Citation

  • Thompson, G., P. R. Field, R.M. Rasmussen, and W. D. Hall, 2008: Explicit forecasts of winter precipitation using an improved bulk microphysics scheme. Part II: Implementation of a new snow parameterization. Mon. Wea. Rev., 136, 5095–5115, doi:10.1175/2008MWR2387.1.

    • Search Google Scholar
    • Export Citation

  • Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. Elsevier, 676 pp.

  • Wolff, J. K., B. S. Ferrier, and C. F. Mass, 2012: Establishing closer collaboration to improve model physics for short-range forecasts. Bull. Amer. Meteor. Soc., 93, ES51–ES53, doi:10.1175/BAMS-D-11-00248.1.

    • Search Google Scholar
    • Export Citation

  • Wolff, J. K., M. Harrold, T. Fowler, J. Halley Gotway, L. Nance, and B. G. Brown, 2014: Beyond the basics: Evaluating model-based precipitation forecasts using traditional, spatial, and object-based methods. Wea. Forecasting, 29, 1451–1472, doi:10.1175/WAF-D-13-00135.1.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 244 107 10
PDF Downloads 104 38 3