Assessment of the High-Resolution Rapid Refresh Model’s Ability to Predict Mesoscale Convective Systems Using Object-Based Evaluation

James O. Pinto Research Applications Laboratory, National Center for Atmospheric Research,* Colorado

Search for other papers by James O. Pinto in
Current site
Google Scholar
PubMed
Close
,
Joseph A. Grim Research Applications Laboratory, National Center for Atmospheric Research,* Colorado

Search for other papers by Joseph A. Grim in
Current site
Google Scholar
PubMed
Close
, and
Matthias Steiner Research Applications Laboratory, National Center for Atmospheric Research,* Colorado

Search for other papers by Matthias Steiner in
Current site
Google Scholar
PubMed
Close
Restricted access

We are aware of a technical issue preventing figures and tables from showing in some newly published articles in the full-text HTML view.
While we are resolving the problem, please use the online PDF version of these articles to view figures and tables.

Abstract

An object-based verification technique that keys off the radar-retrieved vertically integrated liquid (VIL) is used to evaluate how well the High-Resolution Rapid Refresh (HRRR) predicted mesoscale convective systems (MCSs) in 2012 and 2013. It is found that the modeled radar VIL values are roughly 50% lower than observed. This mean bias is accounted for by reducing the radar VIL threshold used to identify MCSs in the HRRR. This allows for a more fair evaluation of the model’s skill at predicting MCSs. Using an optimized VIL threshold for each summer, it is found that the HRRR reproduces the first (i.e., counts) and second moments (i.e., size distribution) of the observed MCS size distribution averaged over the eastern United States, as well as their aspect ratio, orientation, and diurnal variations. Despite threshold optimization, the HRRR tended to predict too many (few) MCSs at lead times less (greater) than 4 h because of lead time–dependent biases in the modeled radar VIL. The HRRR predicted too many MCSs over the Great Plains and too few MCSs over the southeastern United States during the day. These biases are related to the model’s tendency to initiate too many MCSs over the Great Plains and too few MCSs over the southeastern United States. Additional low biases found over the Mississippi River valley region at night revealed a tendency for the HRRR to dissipate MCSs too quickly. The skill of the HRRR at predicting specific MCS events increased between 2012 and 2013, coinciding with changes in both the model physics and in the methods used to assimilate the three-dimensional radar reflectivity.

The National Center for Atmospheric Research is sponsored by the National Science Foundation.

Corresponding author address: James O. Pinto, Research Applications Laboratory, National Center for Atmospheric Research, 3450 Mitchell Ln., Boulder, CO 80301. E-mail: pinto@ucar.edu

Abstract

An object-based verification technique that keys off the radar-retrieved vertically integrated liquid (VIL) is used to evaluate how well the High-Resolution Rapid Refresh (HRRR) predicted mesoscale convective systems (MCSs) in 2012 and 2013. It is found that the modeled radar VIL values are roughly 50% lower than observed. This mean bias is accounted for by reducing the radar VIL threshold used to identify MCSs in the HRRR. This allows for a more fair evaluation of the model’s skill at predicting MCSs. Using an optimized VIL threshold for each summer, it is found that the HRRR reproduces the first (i.e., counts) and second moments (i.e., size distribution) of the observed MCS size distribution averaged over the eastern United States, as well as their aspect ratio, orientation, and diurnal variations. Despite threshold optimization, the HRRR tended to predict too many (few) MCSs at lead times less (greater) than 4 h because of lead time–dependent biases in the modeled radar VIL. The HRRR predicted too many MCSs over the Great Plains and too few MCSs over the southeastern United States during the day. These biases are related to the model’s tendency to initiate too many MCSs over the Great Plains and too few MCSs over the southeastern United States. Additional low biases found over the Mississippi River valley region at night revealed a tendency for the HRRR to dissipate MCSs too quickly. The skill of the HRRR at predicting specific MCS events increased between 2012 and 2013, coinciding with changes in both the model physics and in the methods used to assimilate the three-dimensional radar reflectivity.

The National Center for Atmospheric Research is sponsored by the National Science Foundation.

Corresponding author address: James O. Pinto, Research Applications Laboratory, National Center for Atmospheric Research, 3450 Mitchell Ln., Boulder, CO 80301. E-mail: pinto@ucar.edu
Save
  • Alexander, C., Weygandt S. , Benjamin S. G. , Smirnova T. G. , Brown J. M. , Hofmann P. , and James E. , 2011: The High Resolution Rapid Refresh (HRRR): Recent and future enhancements, time-lagged ensembling, and 2010 forecast evaluation activities. Proc. 24th Conf. on Weather and Forecasting/20th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., 12B.2. [Available online at https://ams.confex.com/ams/91Annual/webprogram/Paper183065.html.]

  • Augustine, J. A., and Howard K. W. , 1988: Mesoscale convective complexes over the United States during 1985. Mon. Wea. Rev., 116, 685701, doi:10.1175/1520-0493(1988)116<0685:MCCOTU>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Baldwin, M. E., and Lakshmivarahan S. , 2003: Development of an events oriented verification system using data mining and image processing algorithms. Preprints, Third Conf. on Artificial Intelligence, Long Beach, CA, Amer. Meteor. Soc., 4.6. [Available online at http://ams.confex.com/ams/pdfpapers/57821.pdf.]

  • Basara, J. B., Maybourn J. N. , Peirano C. M. , Tate J. E. , Brown P. J. , Hoey J. D. , and Smith B. R. , 2013: Drought and associated impacts in the Great Plains of the United States—A review. Int. J. Geosci., 4, 7281, doi:10.4236/ijg.2013.46A2009.

    • Search Google Scholar
    • Export Citation
  • Benjamin, S. G., Weygandt S. , Smirnova T. G. , Hu M. , Peckham S. E. , Brown J. M. , Brundage K. , and Manikin G. S. , 2009: Assimilation of radar reflectivity data using a diabatic digital filter: Applications to the Rapid Update Cycle and Rapid Refresh and initialization of High Resolution Rapid Refresh forecasts with RUC/RR grids. Preprints, 13th Conf. on Integrated Observing and Assimilation Systems for Atmosphere, Oceans, and Land Surface (IOAS-AOLS), Phoenix, AZ, Amer. Meteor. Soc., 7B.3. [Available online at https://ams.confex.com/ams/pdfpapers/150469.pdf.]

  • Benjamin, S. G., Weygandt S. , Alexander C. , Brown J. M. , Smirnova T. G. , Hofmann P. , James E. , and Dimego G. , 2011: NOAA’s hourly-updated 3km HRRR and RUC/Rapid Refresh—Recent (2010) and upcoming changes toward improving weather guidance for air-traffic management. Proc. Second Aviation, Range, and Aerospace Meteorology Special Symp. on Weather–Air Traffic Management Integration, Seattle, WA, Amer. Meteor. Soc., 3.2. [Available online at https://ams.confex.com/ams/91Annual/webprogram/Paper185659.html.]

  • Benjamin, S. G., and Coauthors, 2013: Data assimilation and model updates in the 2013 Rapid Refresh (RAP) and High-Resolution Rapid Refresh (HRRR) analysis and forecast systems. NCEP/EMC Meeting, Washington, DC, NCEP/EMC/Model Evaluation Group. [Available online at http://ruc.noaa.gov/pdf/NCEP_HRRR_RAPv2_6jun2013-Benj-noglob.pdf.]

  • Billet, J., DeLisi M. , Smith B. G. , and Gates C. , 1997: Use of regression techniques to predict hail size and the probability of large hail. Wea. Forecasting, 12, 154164, doi:10.1175/1520-0434(1997)012<0154:UORTTP>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Bryan, G. H., Wyngaard J. C. , and Fritsch J. M. , 2003: Resolution requirements for the simulation of deep moist convection. Mon. Wea. Rev., 131, 23942416, doi:10.1175/1520-0493(2003)131<2394:RRFTSO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Burghardt, B. J., Evans C. , and Roebber P. J. , 2014: Assessing the predictability of convection initiation in the high plains using an object-based approach. Wea. Forecasting, 29, 403418, doi:10.1175/WAF-D-13-00089.1.

    • Search Google Scholar
    • Export Citation
  • Caine, S., Lane T. P. , May P. , Jakob C. , Siems S. T. , Manton M. J. , and Pinto J. O. , 2013: Statistical assessment of tropical convection-permitting model simulations using a cell-tracking algorithm. Mon. Wea. Rev., 141, 557581, doi:10.1175/MWR-D-11-00274.1.

    • Search Google Scholar
    • Export Citation
  • Carbone, R. E., and Tuttle J. D. , 2008: Rainfall occurrence in the U.S. warm season: The diurnal cycle. J. Climate, 21, 41324146, doi:10.1175/2008JCLI2275.1.

    • Search Google Scholar
    • Export Citation
  • Carbone, R. E., Tuttle J. D. , Ahijevych D. , and Trier S. B. , 2002: Inferences of predictability associated with warm season precipitation episodes. J. Atmos. Sci., 59, 20332056, doi:10.1175/1520-0469(2002)059<2033:IOPAWW>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Clark, A. J., Bullock R. G. , Jensen T. L. , Xue M. , and Kong F. , 2014: Application of object-based time-domain diagnostics for tracking precipitation systems in convection allowing models. Wea. Forecasting, 29, 517542, doi:10.1175/WAF-D-13-00098.1.

    • Search Google Scholar
    • Export Citation
  • Coniglio, M. C., Hwang J. Y. , and Stensrud D. J. , 2010: Environmental factors in the upscale growths and longevity of MCSs derived from Rapid Update Cycle analyses. Mon. Wea. Rev., 138, 35143539, doi:10.1175/2010MWR3233.1.

    • Search Google Scholar
    • Export Citation
  • Coniglio, M. C., Correia J. , Marsh P. T. , and Kong F. , 2013: Verification of convection-allowing WRF Model forecasts of the planetary boundary layer using sounding observations. Wea. Forecasting, 28, 842862, doi:10.1175/WAF-D-12-00103.1.

    • Search Google Scholar
    • Export Citation
  • Crowe, B. A., and Miller D. W. , 1999: The benefits of using NEXRAD vertically integrated liquid water as an aviation weather product. Preprints, Eighth Conf. on Aviation, Range, and Aerospace Meteorology, Dallas, TX, Amer. Meteor. Soc., 168171.

  • Davis, C. A., Brown B. G. , and Bullock R. G. , 2006: Object-based verification of precipitation forecasts. Part II: Application to convective rain systems. Mon. Wea. Rev., 134, 17851795, doi:10.1175/MWR3146.1.

    • Search Google Scholar
    • Export Citation
  • Dixon, M., and Wiener G. , 1993: TITAN: Thunderstorm identification, tracking, analysis and nowcasting—A radar-based methodology. J. Atmos. Oceanic Technol., 10, 785797, doi:10.1175/1520-0426(1993)010<0785:TTITAA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., and McBride J. L. , 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239, 179202, doi:10.1016/S0022-1694(00)00343-7.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., and Gallus W. A. Jr., 2009: Toward better understanding of the contiguous rain area (CRA) method for spatial forecast verification. Wea. Forecasting, 24, 14011415, doi:10.1175/2009WAF2222252.1.

    • Search Google Scholar
    • Export Citation
  • Evans, J. E., and Ducot E. R. , 2006: Corridor Integrated Weather System. MIT Lincoln Lab. J., 16, 5980.

  • Gallus, W. A., Jr., 2010: Application of object-based verification techniques to ensemble precipitation forecasts. Wea. Forecasting, 25, 144158, doi:10.1175/2009WAF2222274.1.

    • Search Google Scholar
    • Export Citation
  • Geerts, B., 1998: Mesoscale convective systems in the southeast United States during 1994–95: A survey. Wea. Forecasting, 13, 860869, doi:10.1175/1520-0434(1998)013<0860:MCSITS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Greene, D. R., and Clark R. A. , 1972: Vertically integrated liquid water—A new analysis tool. Mon. Wea. Rev., 100, 548552, doi:10.1175/1520-0493(1972)100<0548:VILWNA>2.3.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hallowell, R. G., and Coauthors, 1999: The Terminal Convective Weather Forecast Demonstration. Preprints, Eighth Conf. on Aviation, Range, and Aerospace Meteorology, Dallas, TX, Amer. Meteor. Soc., 200–204.

  • Hoerling, M., and Coauthors, 2013: An interpretation of the origins of the 2012 Central Great Plains drought. NOAA Drought Task Force Assessment Rep., 50 pp. [Available online at ftp://ftp.oar.noaa.gov/CPO/pdf/mapp/reports/2012-Drought-Interpretation-final.web-041113.pdf.]

  • Hogan, R. J., O’Connor E. J. , and Illingworth A. J. , 2009: Verification of cloud fraction forecasts. Quart. J. Roy. Meteor. Soc., 135, 14941511, doi:10.1002/qj.481.

    • Search Google Scholar
    • Export Citation
  • Houze, R. A., Jr., 1993: Cloud Dynamics. Academic Press, 573 pp.

  • Jirak, I. L., Cotton W. R. , and McAnelly R. L. , 2003: Satellite and radar survey of mesoscale convective system development. Mon. Wea. Rev., 131, 24282449, doi:10.1175/1520-0493(2003)131<2428:SARSOM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Johnson, A., and Wang X. , 2013: Object-based evaluation of a storm-scale ensemble during the 2009 NOAA Hazardous Weather Testbed Spring Experiment. Mon. Wea. Rev., 141, 10791098, doi:10.1175/MWR-D-12-00140.1.

    • Search Google Scholar
    • Export Citation
  • Johnson, A., Wang X. , Kong F. , and Xue M. , 2013: Object-based evaluation of the impact of horizontal grid spacing on convection-allowing forecasts. Mon. Wea. Rev., 141, 34133425, doi:10.1175/MWR-D-13-00027.1.

    • Search Google Scholar
    • Export Citation
  • Jolliffe, I. T., 2008: The impenetrable hedge: A note on propriety, equitability and consistency. Meteor. Appl., 15, 2529, doi:10.1002/met.60.

    • Search Google Scholar
    • Export Citation
  • Kitzmiller, D. H., McGovern W. E. , and Saffle R. F. , 1995: The WSR-88D severe weather potential algorithm. Wea. Forecasting, 10, 141159, doi:10.1175/1520-0434(1995)010<0141:TWSWPA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Koch, S. E., Ferrier B. S. , Kain J. S. , Stoelinga M. T. , Szoke E. J. , and Weiss S. J. , 2005: The use of simulated radar reflectivity fields in the diagnosis of mesoscale phenomena from high-resolution WRF Model forecasts. Preprints, 11th Conf. on Mesoscale Processes/32nd Conf. on Radar Meteorology, Albuquerque, NM, Amer. Meteor. Soc., J4J.7. [Available online at https://ams.confex.com/ams/pdfpapers/97032.pdf.]

  • Krozel, J., Mitchell J. S. B. , Polishchuk V. , and Prete J. , 2007: Maximum flow rates for capacity estimation in level flight with convective weather constraints. Air Traffic Control Quart., 15, 209238.

    • Search Google Scholar
    • Export Citation
  • Lack, S., Limpert G. J. , and Fox N. I. , 2010: An object-oriented multiscale verification scheme. Wea. Forecasting, 25, 7992, doi:10.1175/2009WAF2222245.1.

    • Search Google Scholar
    • Export Citation
  • Parker, M. D., and Johnson R. H. , 2000: Organizational modes of midlatitude mesoscale convective systems. Mon. Wea. Rev., 128, 34133436, doi:10.1175/1520-0493(2001)129<3413:OMOMMC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Parker, M. D., and Ahijevych D. , 2007: Convective episodes in the east-central United States. Mon. Wea. Rev., 135, 37073727, doi:10.1175/2007MWR2098.1.

    • Search Google Scholar
    • Export Citation
  • Pinto, J. O., Phillips C. , Steiner M. , Rasmussen R. , Oien N. , Dixon M. , Wang W. , and Weisman M. , 2007: Assessment of the statistical characteristics of thunderstorms simulated with the WRF Model using convection permitting resolution. 33rd Int. Conf. on Radar Meteorology, Cairns, QLD, Australia, 5.5. [Available online at https://ams.confex.com/ams/pdfpapers/123712.pdf.]

  • Pinto, J. O., Dupree W. , Weygandt S. , Wolfson M. , Benjamin S. , and Steiner M. , 2010: Advances in the Consolidated Storm Prediction for Aviation (CoSPA). Preprints, 14th Conf. on Aviation, Range, and Aerospace Meteorology, Atlanta, GA, Amer. Meteor. Soc., J11.2. [Available online at https://ams.confex.com/ams/pdfpapers/163811.pdf.]

  • Pinto, J. O., Grim J. A. , Ahijevych D. , and Steiner M. , 2013: An automated system for detecting large-scale convective storms: Application to model evaluation. Proc. 16th Conf. on Aviation, Range, and Aerospace Meteorology, Austin, TX, Amer. Meteor. Soc., 9.4A. [Available online at https://ams.confex.com/ams/93Annual/webprogram/Paper222079.html].

  • Schwartz, C. S., and Coauthors, 2009: Next-day convection-allowing WRF Model guidance: A second look at 2-km versus 4-km grid spacing. Mon. Wea. Rev., 137, 33513372, doi:10.1175/2009MWR2924.1.

    • Search Google Scholar
    • Export Citation
  • Skamarock, W. C., and Klemp J. B. , 2008: A time-split nonhydrostatic atmosphere model for weather research and forecasting applications. J. Comput. Phys., 227, 34653485, doi:10.1016/j.jcp.2007.01.037.

    • Search Google Scholar
    • Export Citation
  • Smalley, D. J., and Bennett B. J. , 2002: Using ORPG to enhance NEXRAD products to support FAA critical systems. Preprints, 10th Conf. on Aviation, Range, and Aerospace Meteorology, Portland, OR, Amer. Meteor. Soc., 3.6. [Available online at https://ams.confex.com/ams/pdfpapers/38861.pdf.]

  • Steiner, M., Bateman R. , Megenhardt D. , Liu Y. , Xu M. , Pocernich M. , and Krozel J. , 2010: Translation of ensemble weather forecasts into probabilistic air traffic capacity impact. Air Traffic Control Quart., 18, 229254.

    • Search Google Scholar
    • Export Citation
  • Stensrud, D. J., and Coauthors, 2009: Convective-scale warn-on-forecast: A vision for 2020. Bull. Amer. Meteor. Soc., 90, 14871499, doi:10.1175/2009BAMS2795.1.

    • Search Google Scholar
    • Export Citation
  • Stensrud, D. J., and Coauthors, 2013: Progress and challenges with warn-on-forecast. Atmos. Environ., 123, 216, doi:10.1016/j.atmosres.2012.04.004.

    • Search Google Scholar
    • Export Citation
  • Stratman, D. R., Coniglio M. C. , Koch S. E. , and Xue M. , 2013: Use of multiple verification methods to evaluation forecasts of convection from hot- and cold-start convection allowing models. Wea. Forecasting, 28, 119138, doi:10.1175/WAF-D-12-00022.1.

    • Search Google Scholar
    • Export Citation
  • Thompson, G., Field P. R. , Rasmussen R. M. , and Hall W. D. , 2008: Explicit forecasts of winter precipitation using an improved bulk microphysics scheme: Part II: Implementation of a new snow parameterization. Mon. Wea. Rev., 136, 50955115, doi:10.1175/2008MWR2387.1.

    • Search Google Scholar
    • Export Citation
  • Weisman, M. L., Skamarock W. C. , and Klemp J. B. , 1997: The resolution dependence of explicitly modeled convective systems. Mon. Wea. Rev., 125, 527548, doi:10.1175/1520-0493(1997)125<0527:TRDOEM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Weygandt, S., and Coauthors, 2011: The Rapid Refresh—Replacement for the RUC, pre-implementation development and evaluation. Proc. 24th Conf. on Weather and Forecasting/20th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., 12B.1. [Available online at https://ams.confex.com/ams/91Annual/webprogram/Paper183027.html.]

  • Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. 2nd ed. Academic Press, 648 pp.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1151 569 178
PDF Downloads 593 129 8