Editorial Type:
Article
Article Type:
Research Article

Methods for Validating HRRR Simulated Cloud Properties for Different Weather Phenomena Using Satellite and Radar Observations

Sarah M. Griffin aCooperative Institute for Meteorological Satellite Studies, University of Wisconsin–Madison, Madison, Wisconsin

Search for other papers by Sarah M. Griffin in
Current site
Google Scholar
PubMed Close
https://orcid.org/0000-0002-5089-5274
,
Jason A. Otkin aCooperative Institute for Meteorological Satellite Studies, University of Wisconsin–Madison, Madison, Wisconsin

Search for other papers by Jason A. Otkin in
Current site
Google Scholar
PubMed Close
, and
William E. Lewis aCooperative Institute for Meteorological Satellite Studies, University of Wisconsin–Madison, Madison, Wisconsin

Search for other papers by William E. Lewis in
Current site
Google Scholar
PubMed Close
Online Publication:
22 Dec 2023
Print Publication:
01 Jan 2024
DOI:
https://doi.org/10.1175/WAF-D-23-0109.1
Page(s):
97–120
Restricted access

Abstract

In this study, we evaluate the ability of the High-Resolution Rapid Refresh (HRRR) model to forecast cloud characteristics through comparison of observed and simulated satellite brightness temperatures (BTs) and radar reflectivity during different weather phenomena in December 2021: the Mayfield, Kentucky, tornado on 11 December, a heavy snow event in Minnesota from 10 to 11 December, and the Midwest derecho on 15 December. This is done to illustrate the importance of examining model accuracy across a range of weather phenomena. Observation and forecast objects were created using the Method for Object-Based Diagnostic Evaluation (MODE). HRRR accurately depicted the spatial displacements between observation cloud (defined using BTs) and radar reflectivity objects, namely, the centers of cloud objects are to the east of the radar objects for the tornado and derecho events, and generally west of the radar objects for the snow event. However, HRRR had higher (less intense) simulated BTs and higher (more intense) radar reflectivity than the observations for the tornado event. Simulated radar reflectivity is higher and BTs are lower than the observations during the middle of the snow event. Also, simulated radar reflectivity is higher and BTs are lower than the observations during the derecho event. Of the three weather events, the HRRR forecasts are most accurate for the snow event, based on the object-based threat score, followed by the derecho and tornado events. The tornado event has lower accuracy because matches between paired simulated and observation objects are worse than for the snow event, with less similarity in size forecast objects and greater distance between paired object centers.

Significance Statement

The purpose of this study is to assess the accuracy of forecast cloud and radar objects, defined using simulated satellite brightness temperatures and radar reflectivity, from the High-Resolution Rapid Refresh (HRRR) model. This assessment was conducted for a tornado, snow, and derecho event from December 2021. Results from these three events indicate that the HRRR model accurately represents the observed displacement between the center of cloud and radar objects for the tornado and derecho events, and is the most accurate overall for the snow event.

© 2023 American Meteorological Society. This published article is licensed under the terms of the default AMS reuse license. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Sarah M. Griffin, sarah.griffin@ssec.wisc.edu

Abstract

In this study, we evaluate the ability of the High-Resolution Rapid Refresh (HRRR) model to forecast cloud characteristics through comparison of observed and simulated satellite brightness temperatures (BTs) and radar reflectivity during different weather phenomena in December 2021: the Mayfield, Kentucky, tornado on 11 December, a heavy snow event in Minnesota from 10 to 11 December, and the Midwest derecho on 15 December. This is done to illustrate the importance of examining model accuracy across a range of weather phenomena. Observation and forecast objects were created using the Method for Object-Based Diagnostic Evaluation (MODE). HRRR accurately depicted the spatial displacements between observation cloud (defined using BTs) and radar reflectivity objects, namely, the centers of cloud objects are to the east of the radar objects for the tornado and derecho events, and generally west of the radar objects for the snow event. However, HRRR had higher (less intense) simulated BTs and higher (more intense) radar reflectivity than the observations for the tornado event. Simulated radar reflectivity is higher and BTs are lower than the observations during the middle of the snow event. Also, simulated radar reflectivity is higher and BTs are lower than the observations during the derecho event. Of the three weather events, the HRRR forecasts are most accurate for the snow event, based on the object-based threat score, followed by the derecho and tornado events. The tornado event has lower accuracy because matches between paired simulated and observation objects are worse than for the snow event, with less similarity in size forecast objects and greater distance between paired object centers.

Significance Statement

The purpose of this study is to assess the accuracy of forecast cloud and radar objects, defined using simulated satellite brightness temperatures and radar reflectivity, from the High-Resolution Rapid Refresh (HRRR) model. This assessment was conducted for a tornado, snow, and derecho event from December 2021. Results from these three events indicate that the HRRR model accurately represents the observed displacement between the center of cloud and radar objects for the tornado and derecho events, and is the most accurate overall for the snow event.

© 2023 American Meteorological Society. This published article is licensed under the terms of the default AMS reuse license. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Sarah M. Griffin, sarah.griffin@ssec.wisc.edu

Supplementary Materials

    • Supplemental Materials (PDF 0.5740 MB)
Save
Cover Weather and Forecasting
Weather and Forecasting

  • Bedka, K., J. Brunner, R. Dworak, W. Feltz, J. Otkin, and T. Greenwald, 2010: Objective satellite-based detection of overshooting tops using infrared window channel brightness temperature gradients. J. Appl. Meteor. Climatol., 49, 181–202, https://doi.org/10.1175/2009JAMC2286.1.

    • Search Google Scholar
    • Export Citation

  • Bikos, D., and Coauthors, 2012: Synthetic satellite imagery for real-time high-resolution model evaluation. Wea. Forecasting, 27, 784–795, https://doi.org/10.1175/WAF-D-11-00130.1.

    • Search Google Scholar
    • Export Citation

  • Blahak, U., 2012: RADAR_MIE_LM and RADAR_MIELIB—Calculation of radar reflectivity from model output. Tech. Doc., 167 pp., https://download.dwd.de/pub/DWD/Forschung_und_Entwicklung/RADVOP/mielib.pdf.

  • Blank, L. R., R. Bullock, D. Fillmore, J. Halley Gotway, T. L. Jensen, and J. K. Wolff, 2021: Automated identification of drylines in the United States using multivariate MODE. 11th Conf. on Transition of Research to Operations, online, Amer. Meteor. Soc., 5A.9, https://ams.confex.com/ams/101ANNUAL/meetingapp.cgi/Paper/383663.

  • Brooks, H. E., C. A. Doswell III, and R. B. Wilhelmson, 1994: The role of midtropospheric winds in the evolution and maintenance of low-level mesocyclones. Mon. Wea. Rev., 122, 126–136, https://doi.org/10.1175/1520-0493(1994)122<0126:TROMWI>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Bullock, R. G., B. G. Brown, and T. L. Fowler, 2016: Method for object-based diagnostic evaluation. NCAR Tech. Note NCAR/TN-532+STR, 84 pp., https://doi.org/10.5065/D61V5CBS.

  • Bytheway, J. L., and C. D. Kummerow, 2015: Toward an object-based assessment of high-resolution forecasts of long-lived convective precipitation in the central U.S. J. Adv. Model. Earth Syst., 7, 1248–1264, https://doi.org/10.1002/2015MS000497.

    • Search Google Scholar
    • Export Citation

  • Bytheway, J. L., C. D. Kummerow, and C. Alexander, 2017: A features-based assessment of the evolution of warm season precipitation forecasts from the HRRR Model over three years of development. Wea. Forecasting, 32, 1841–1856, https://doi.org/10.1175/WAF-D-17-0050.1.

    • Search Google Scholar
    • Export Citation

  • Cai, H., and R. E. Dumais Jr., 2015: Object-based evaluation of a numerical weather prediction model’s performance through forecast storm characteristic analysis. Wea. Forecasting, 30, 1451–1468, https://doi.org/10.1175/WAF-D-15-0008.1.

    • Search Google Scholar
    • Export Citation

  • Cintineo, R., J. A. Otkin, M. Xue, and F. Kong, 2014: Evaluating the performance of planetary boundary layer and cloud microphysical parameterization schemes in convection-permitting ensemble forecasts using synthetic GOES-13 satellite observations. Mon. Wea. Rev., 142, 163–182, https://doi.org/10.1175/MWR-D-13-00143.1.

    • Search Google Scholar
    • Export Citation

  • Davis, C., B. Brown, and R. Bullock, 2006a: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Wea. Rev., 134, 1772–1784, https://doi.org/10.1175/MWR3145.1.

    • Search Google Scholar
    • Export Citation

  • Davis, C., B. Brown, and R. Bullock, 2006b: Object-based verification of precipitation forecasts. Part II: Application to convective rain systems. Mon. Wea. Rev., 134, 1785–1795, https://doi.org/10.1175/MWR3146.1.

    • Search Google Scholar
    • Export Citation

  • Davis, C., B. Brown, R. Bullock, and J. Halley-Gotway, 2009: The Method for Object-based Diagnostic Evaluation (MODE) applied to numerical forecasts from the 2005 NSSL/SPC spring program. Wea. Forecasting, 24, 1252–1267, https://doi.org/10.1175/2009WAF2222241.1.

    • Search Google Scholar
    • Export Citation

  • DeHaan, L. L., A. C. Martin, R. R. Weihs, L. Delle Monache, and F. M. Ralph, 2021: Object-based verification of atmospheric river predictions in the northeast Pacific. Wea. Forecasting, 36, 1575–1587, https://doi.org/10.1175/WAF-D-20-0236.1.

    • Search Google Scholar
    • Export Citation

  • Ding, S., P. Yang, F. Weng, Q. Liu, Y. Han, P. Van Delst, J. Li, and B. Baum, 2011: Validation of the community radiative transfer model. J. Quant. Spectrosc. Radiat. Transfer, 112, 1050–1064, https://doi.org/10.1016/j.jqsrt.2010.11.009.

    • Search Google Scholar
    • Export Citation

  • Dougherty, K. J., J. D. Horel, and J. E. Nachamkin, 2021: Forecast skill for California heavy precipitation periods from the High-Resolution Rapid Refresh Model and the coupled ocean–atmosphere mesoscale prediction system. Wea. Forecasting, 36, 2275–2288, https://doi.org/10.1175/WAF-D-20-0182.1.

    • Search Google Scholar
    • Export Citation

  • Duda, J. D., and D. D. Turner, 2021: Large-sample application of radar reflectivity object-based verification to evaluate HRRR warm-season forecasts. Wea. Forecasting, 36, 805–821, https://doi.org/10.1175/WAF-D-20-0203.1.

    • Search Google Scholar
    • Export Citation

  • English, J. M., D. D. Turner, T. I. Alcott, W. R. Moninger, J. L. Bytheway, R. Cifelli, and M. Marquis, 2021: Evaluating operational and experimental HRRR model forecasts of atmospheric river events in California. Wea. Forecasting, 36, 1925–1944, https://doi.org/10.1175/WAF-D-21-0081.1.

    • Search Google Scholar
    • Export Citation

  • Ganetis, S. A., B. A. Colle, S. E. Yuter, and N. P. Hoban, 2018: Environmental conditions associated with observed snowband structures within northeast U.S. winter storms. Mon. Wea. Rev., 146, 3675–3690, https://doi.org/10.1175/MWR-D-18-0054.1.

    • Search Google Scholar
    • Export Citation

  • Griffin, S. M., J. A. Otkin, C. M. Rozoff, J. M. Sieglaff, L. M. Cronce, and C. R. Alexander, 2017a: Methods for comparing simulated and observed satellite infrared brightness temperatures and what do they tell us? Wea. Forecasting, 32, 5–25, https://doi.org/10.1175/WAF-D-16-0098.1.

    • Search Google Scholar
    • Export Citation

  • Griffin, S. M., J. A. Otkin, C. M. Rozoff, J. M. Sieglaff, L. M. Cronce, C. R. Alexander, T. L. Jensen, and J. K. Wolff, 2017b: Seasonal analysis of cloud objects in the High-Resolution Rapid Refresh (HRRR) Model using object-based verification. J. Appl. Meteor. Climatol., 56, 2317–2334, https://doi.org/10.1175/JAMC-D-17-0004.1.

    • Search Google Scholar
    • Export Citation

  • Griffin, S. M., J. A. Otkin, G. Thompson, M. Frediani, J. Berner, and F. Kong, 2020: Assessing the impact of stochastic perturbations in cloud microphysics using GOES-16 infrared brightness temperatures. Mon. Wea. Rev., 148, 3111–3137, https://doi.org/10.1175/MWR-D-20-0078.1.

    • Search Google Scholar
    • Export Citation

  • Griffin, S. M., J. A. Otkin, S. E. Nebuda, T. L. Jensen, P. S. Skinner, E. Gilleland, T. A. Supinie, and M. Xue, 2021: Evaluating the impact of planetary boundary layer, land surface model, and microphysics parameterization schemes on cold cloud objects in simulated GOES-16 brightness temperatures. J. Geophys. Res. Atmos., 126, e2021JD034709, https://doi.org/10.1029/2021JD034709.

    • Search Google Scholar
    • Export Citation

  • Han, Y., P. van Delst, Q. Liu, F. Weng, B. Yan, R. Treadon, and J. Derber, 2006: JCSDA Community Radiative Transfer Model (CRTM)—Version 1. NOAA Tech. Rep. NESDIS 122, 33 pp., https://www.star.nesdis.noaa.gov/pub/smcd/jcsda/qliu/REL_2.1.3_June21_2013/CRTM_v1-NOAA_Tech_Report_NESDIS122.pdf.

  • Hanna, J. W., D. M. Schultz, and A. R. Irving, 2008: Cloud-top temperatures for precipitating winter clouds. J. Appl. Meteor. Climatol., 47, 351–359, https://doi.org/10.1175/2007JAMC1549.1.

    • Search Google Scholar
    • Export Citation

  • Heidinger, A., and Y. Li, 2018: AWG Cloud Height Algorithm (ACHA). NOAA NESDIS Center for Satellite Applications and Research Algorithm Theoretical Basis Doc., version 3.2, 60 pp., https://www.ssec.wisc.edu/∼daves/ACHA_ATBD.pdf.

  • Henderson, D. S., J. A. Otkin, and J. R. Mecikalski, 2021: Evaluating convective initiation in high-resolution numerical weather prediction models using GOES-16 infrared brightness temperatures. Mon. Wea. Rev., 149, 1153–1172, https://doi.org/10.1175/MWR-D-20-0272.1.

    • Search Google Scholar
    • Export Citation

  • Iacono, M. J., J. S. Delamere, E. J. Mlawer, M. W. Shephard, S. A. Clough, and W. D. Collins, 2008: Radiative forcing by long-lived greenhouse gases: Calculations with the AER radiative transfer models. J. Geophys. Res., 113, D13103, https://doi.org/10.1029/2008JD009944.

    • Search Google Scholar
    • Export Citation

  • Ikeda, K., M. Steiner, J. Pinto, and C. Alexander, 2013: Evaluation of cold-season precipitation forecasts generated by the hourly updating High-Resolution Rapid Refresh Model. Wea. Forecasting, 28, 921–939, https://doi.org/10.1175/WAF-D-12-00085.1.

    • Search Google Scholar
    • Export Citation

  • Ikeda, K., M. Steiner, and G. Thompson, 2017: Examination of mixed-phase precipitation forecasts from the High-Resolution Rapid Refresh Model using surface observations and sounding data. Wea. Forecasting, 32, 949–967, https://doi.org/10.1175/WAF-D-16-0171.1.

    • Search Google Scholar
    • Export Citation

  • James, E. P., and Coauthors, 2022: The High-Resolution Rapid Refresh (HRRR): An hourly updating convection-allowing forecast model. Part II: Forecast performance. Wea. Forecasting, 37, 1397–1417, https://doi.org/10.1175/WAF-D-21-0130.1.

    • Search Google Scholar
    • Export Citation

  • Jensen, T., B. Brown, R. Bullock, T. Fowler, J. Halley Gotway, and K. Newman, 2020: Model evaluation tools version 9.0.1 user’s guide. Developmental Testbed Center, 482 pp., https://dtcenter.org/sites/default/files/community-code/met/docs/user-guide/MET_Users_Guide_v9.0.1.pdf.

  • Ji, L., X. Zhi, C. Simmer, S. Zhu, and Y. Ji, 2020: Multimodel ensemble forecasts of precipitation based on an object-based diagnostic evaluation. Mon. Wea. Rev., 148, 2591–2606, https://doi.org/10.1175/MWR-D-19-0266.1.

    • Search Google Scholar
    • Export Citation

  • Johnson, A., and X. Wang, 2013: Object-based evaluation of a storm-scale ensemble during the 2009 NOAA Hazardous Weather Testbed Spring Experiment. Mon. Wea. Rev., 141, 1079–1098, https://doi.org/10.1175/MWR-D-12-00140.1.

    • Search Google Scholar
    • Export Citation

  • Jones, T. A., P. Skinner, K. Knopmeier, E. Mansell, P. Minnis, R. Palikonda, and W. Smith Jr., 2018: Comparison of cloud microphysics schemes in a Warn-on-Forecast System using synthetic satellite objects. Wea. Forecasting, 33, 1681–1708, https://doi.org/10.1175/WAF-D-18-0112.1.

    • Search Google Scholar
    • Export Citation

  • Jones, T. A., and Coauthors, 2020: Assimilation of GOES-16 radiances and retrievals into the Warn-on-Forecast System. Mon. Wea. Rev., 148, 1829–1859, https://doi.org/10.1175/MWR-D-19-0379.1.

    • Search Google Scholar
    • Export Citation

  • Nakanishi, M., and H. Niino, 2004: An improved Mellor–Yamada Level-3 model with condensation physics: Its design and verification. Bound.-Layer Meteor., 112, 1–31, https://doi.org/10.1023/B:BOUN.0000020164.04146.98.

    • Search Google Scholar
    • Export Citation

  • Nakanishi, M., and H. Niino, 2009: Development of an improved turbulence closure model for the atmospheric boundary layer. J. Meteor. Soc. Japan, 87, 895–912.

    • Search Google Scholar
    • Export Citation

  • Otkin, J. A., T. J. Greenwald, J. Sieglaff, and H.-L. Huang, 2009: Validation of a large-scale simulated brightness temperature dataset using SEVIRI satellite observations. J. Appl. Meteor. Climatol., 48, 1613–1626, https://doi.org/10.1175/2009JAMC2142.1.

    • Search Google Scholar
    • Export Citation

  • Pinto, J. O., J. A. Grim, and M. Steiner, 2015: Assessment of the High-Resolution Rapid Refresh Model’s ability to predict mesoscale convective systems using object-based evaluation. Wea. Forecasting, 30, 892–913, https://doi.org/10.1175/WAF-D-14-00118.1.

    • Search Google Scholar
    • Export Citation

  • Powers, J. G., and Coauthors, 2017: The Weather Research and Forecasting Model: Overview, system efforts, and future directions. Bull. Amer. Meteor. Soc., 98, 1717–1737, https://doi.org/10.1175/BAMS-D-15-00308.1.

    • Search Google Scholar
    • Export Citation

  • Radford, J. T., G. M. Lackmann, and M. A. Baxter, 2019: An evaluation of snowband predictability in the High-Resolution Rapid Refresh. Wea. Forecasting, 34, 1477–1494, https://doi.org/10.1175/WAF-D-19-0089.1.

    • Search Google Scholar
    • Export Citation

  • Rasmussen, E. N., and J. M. Straka, 1998: Variations in supercell morphology. Part I: Observations of the role of upper-level storm-relative flow. Mon. Wea. Rev., 126, 2406–2421, https://doi.org/10.1175/1520-0493(1998)126<2406:VISMPI>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Senf, F., and H. Deneke, 2017: Uncertainties in synthetic Meteosat SEVIRI infrared brightness temperatures in the presence of cirrus clouds and implications for evaluation of cloud microphysics. Atmos. Res., 183, 113–129, https://doi.org/10.1016/j.atmosres.2016.08.012.

    • Search Google Scholar
    • Export Citation

  • Senf, F., D. Klocke, and M. Brueck, 2018: Size-resolved evaluation of simulated deep tropical convection. Mon. Wea. Rev., 146, 2161–2182, https://doi.org/10.1175/MWR-D-17-0378.1.

    • Search Google Scholar
    • Export Citation

  • Skinner, P. S., L. J. Wicker, D. M. Wheatley, and K. H. Knopfmeier, 2016: Application of two spatial verification methods to ensemble forecasts of low-level rotation. Wea. Forecasting, 31, 713–735, https://doi.org/10.1175/WAF-D-15-0129.1.

    • Search Google Scholar
    • Export Citation

  • Skinner, P. S., and Coauthors, 2018: Object-based verification of a prototype Warn-on-Forecast System. Wea. Forecasting, 33, 1225–1250, https://doi.org/10.1175/WAF-D-18-0020.1.

    • Search Google Scholar
    • Export Citation

  • Smirnova, T. G., J. M. Brown, S. G. Benjamin, and J. S. Kenyon, 2016: Modifications to the Rapid Update Cycle Land Surface Model (RUC LSM) available in the Weather Research and Forecasting (WRF) Model. Mon. Wea. Rev., 144, 1851–1865, https://doi.org/10.1175/MWR-D-15-0198.1.

    • Search Google Scholar
    • Export Citation

  • Smith, T. M., and Coauthors, 2016: Multi-Radar Multi-Sensor (MRMS) severe weather and aviation products: Initial operating capabilities. Bull. Amer. Meteor. Soc., 97, 1617–1630, https://doi.org/10.1175/BAMS-D-14-00173.1.

    • Search Google Scholar
    • Export Citation

  • Squitieri, B. J., and W. A. Gallus Jr., 2020: On the forecast sensitivity of MCS cold pools and related features to horizontal grid spacing in convection-allowing WRF simulations. Wea. Forecasting, 35, 325–346, https://doi.org/10.1175/WAF-D-19-0016.1.

    • Search Google Scholar
    • Export Citation

  • Thompson, G., and T. Eidhammer, 2014: A study of aerosol impacts on clouds and precipitation development in a large winter cyclone. J. Atmos. Sci., 71, 3636–3658, https://doi.org/10.1175/JAS-D-13-0305.1.

    • Search Google Scholar
    • Export Citation

  • Thompson, G., M. Tewari, K. Ikeda, S. Tessendorf, C. Weeks, J. Otkin, and F. Kong, 2016: Explicitly-coupled cloud physics and radiation parameterizations and subsequent evaluation in WRF high-resolution convective forecasts. Atmos. Res., 168, 92–104, https://doi.org/10.1016/j.atmosres.2015.09.005.

    • Search Google Scholar
    • Export Citation

  • Van Weverberg, K., and Coauthors, 2013: The role of cloud microphysics parameterization in the simulation of mesoscale convective system clouds and precipitation in the tropical western Pacific. J. Atmos. Sci., 70, 1104–1128, https://doi.org/10.1175/JAS-D-12-0104.1.

    • Search Google Scholar
    • Export Citation

  • Wolff, J. K., M. Harrold, T. Fowler, J. H. Gotway, L. Nance, and B. G. Brown, 2014: Beyond the basics: Evaluating model-based precipitation forecasts using traditional, spatial, and object-based methods. Wea. Forecasting, 29, 1451–1472, https://doi.org/10.1175/WAF-D-13-00135.1.

    • Search Google Scholar
    • Export Citation

  • Zhang, J., and Coauthors, 2016: Multi-Radar Multi-Sensor (MRMS) quantitative precipitation estimation: Initial operating capabilities. Bull. Amer. Meteor. Soc., 97, 621–638, https://doi.org/10.1175/BAMS-D-14-00174.1.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 568 568 58
Full Text Views 191 191 9
PDF Downloads 201 201 10