Editorial Type:
Article
Article Type:
Research Article

On the Importance of Regime-Specific Evaluations for Numerical Weather Prediction Models as Demonstrated Using the High-Resolution Rapid Refresh (HRRR) Model

Temple R. Lee aNOAA/Air Resources Laboratory, Oak Ridge, Tennessee

Search for other papers by Temple R. Lee in
Current site
Google Scholar
PubMed Close
https://orcid.org/0000-0001-5388-6325
,
Sandip Pal bAtmospheric Science Group, Department of Geosciences, Texas Tech University, Lubbock, Texas

Search for other papers by Sandip Pal in
Current site
Google Scholar
PubMed Close
https://orcid.org/0000-0001-9497-9990
,
Ronald D. Leeper cNorth Carolina Institute for Climate Studies, Asheville, North Carolina
dNOAA/National Centers for Environmental Information, Asheville, North Carolina
eCenter for Weather and Climate, Asheville, North Carolina

Search for other papers by Ronald D. Leeper in
Current site
Google Scholar
PubMed Close
,
Tim Wilson aNOAA/Air Resources Laboratory, Oak Ridge, Tennessee
fOak Ridge Associated Universities, Oak Ridge, Tennessee

Search for other papers by Tim Wilson in
Current site
Google Scholar
PubMed Close
,
Howard J. Diamond gNOAA/Air Resources Laboratory, College Park, Maryland

Search for other papers by Howard J. Diamond in
Current site
Google Scholar
PubMed Close
,
Tilden P. Meyers hNOAA/Air Resources Laboratory, Boulder, Colorado

Search for other papers by Tilden P. Meyers in
Current site
Google Scholar
PubMed Close
, and
David D. Turner iNOAA/Global Systems Laboratory, Boulder, Colorado

Search for other papers by David D. Turner in
Current site
Google Scholar
PubMed Close
Online Publication:
21 May 2024
Print Publication:
01 May 2024
DOI:
https://doi.org/10.1175/WAF-D-23-0177.1
Page(s):
781–791
Open access
View license

We are aware of a technical issue preventing figures and tables from showing in some newly published articles in the full-text HTML view.
While we are resolving the problem, please use the online PDF version of these articles to view figures and tables.

Abstract

The scientific literature has many studies evaluating numerical weather prediction (NWP) models. However, many of those studies averaged across a myriad of different atmospheric conditions and surface forcings that can obfuscate the atmospheric conditions when NWP models perform well versus when they perform inadequately. To help isolate these different weather conditions, we used observations from the U.S. Climate Reference Network (USCRN) obtained between 1 January and 31 December 2021 to distinguish among different near-surface atmospheric conditions [i.e., different near-surface heating rates (dT/dt), incoming shortwave radiation (SWd) regimes, and 5-cm soil moisture (SM05)] to evaluate the High-Resolution Rapid Refresh (HRRR) Model, which is a 3-km model used for operational weather forecasting in the United States. On days with small (large) dT/dt, we found afternoon T biases of about 2°C (−1°C) and afternoon SWd biases of up to 170 W m−2 (100 W m−2), but negligible impacts on SM05 biases. On days with small (large) SWd, we found daytime temperature biases of about 3°C (−2.5°C) and daytime SWd biases of up to 190 W m−2 (80 W m−2). Whereas different SM05 had little impact on T and SWd biases, dry (wet) conditions had positive (negative) SM05 biases. We argue that the proper evaluation of weather forecasting models requires careful consideration of different near-surface atmospheric conditions and is critical to better identify model deficiencies in order to support improvements to the parameterization schemes used therein. A similar, regime-specific verification approach may also be used to help evaluate other geophysical models.

Significance Statement

Improving weather forecasting models requires careful evaluations against high-quality observations. We used observations from the U.S. Climate Reference Network (USCRN) and found that the performance of the High-Resolution Rapid Refresh (HRRR) Model varies as a function of differences in near-surface heating and solar radiation. This finding indicates that model evaluations need to be conducted under varying near-surface weather conditions rather than averaging across multiple weather types. This new approach will allow for model developers to better identify model deficiencies and is a useful step to helping improve weather forecasts.

© 2024 American Meteorological Society. This published article is licensed under the terms of the default AMS reuse license. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Temple R. Lee, temple.lee@noaa.gov

Abstract

The scientific literature has many studies evaluating numerical weather prediction (NWP) models. However, many of those studies averaged across a myriad of different atmospheric conditions and surface forcings that can obfuscate the atmospheric conditions when NWP models perform well versus when they perform inadequately. To help isolate these different weather conditions, we used observations from the U.S. Climate Reference Network (USCRN) obtained between 1 January and 31 December 2021 to distinguish among different near-surface atmospheric conditions [i.e., different near-surface heating rates (dT/dt), incoming shortwave radiation (SWd) regimes, and 5-cm soil moisture (SM05)] to evaluate the High-Resolution Rapid Refresh (HRRR) Model, which is a 3-km model used for operational weather forecasting in the United States. On days with small (large) dT/dt, we found afternoon T biases of about 2°C (−1°C) and afternoon SWd biases of up to 170 W m−2 (100 W m−2), but negligible impacts on SM05 biases. On days with small (large) SWd, we found daytime temperature biases of about 3°C (−2.5°C) and daytime SWd biases of up to 190 W m−2 (80 W m−2). Whereas different SM05 had little impact on T and SWd biases, dry (wet) conditions had positive (negative) SM05 biases. We argue that the proper evaluation of weather forecasting models requires careful consideration of different near-surface atmospheric conditions and is critical to better identify model deficiencies in order to support improvements to the parameterization schemes used therein. A similar, regime-specific verification approach may also be used to help evaluate other geophysical models.

Significance Statement

Improving weather forecasting models requires careful evaluations against high-quality observations. We used observations from the U.S. Climate Reference Network (USCRN) and found that the performance of the High-Resolution Rapid Refresh (HRRR) Model varies as a function of differences in near-surface heating and solar radiation. This finding indicates that model evaluations need to be conducted under varying near-surface weather conditions rather than averaging across multiple weather types. This new approach will allow for model developers to better identify model deficiencies and is a useful step to helping improve weather forecasts.

© 2024 American Meteorological Society. This published article is licensed under the terms of the default AMS reuse license. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Temple R. Lee, temple.lee@noaa.gov
Keywords: Atmosphere-land interaction; Shortwave radiation; Soil moisture; Surface observations; Model evaluation/performance

1. Introduction

Evaluating operational numerical weather prediction (NWP) models is critical for identifying weather conditions in which the models perform well in addition to the scenarios where the models perform inadequately. Knowledge of the conditional nature of these biases helps the meteorological community understand, modify, and improve the physical parameterization schemes used therein, which we anticipate will lead to more accurate weather forecasts. Oftentimes when model evaluations are conducted many different atmospheric conditions, ranging from highly convective to very stable conditions, are averaged together. For example, the High-Resolution Rapid Refresh (HRRR), which is a 3-km convection-allowing model that for nearly 10 years has been used to support operational weather forecasting needs in the United States (e.g., Benjamin et al. 2016; Dowell et al. 2022; James et al. 2022), has previously been evaluated across multiple months and seasons. These studies identified biases between the model and observations and indicated several model–data mismatches (MDMs) in, for example the HRRR’s near- and subsurface meteorological fields (e.g., Min et al. 2021; Patel et al. 2021; Lee et al. 2023, hereafter L23), surface fluxes (e.g., Lee et al. 2019; He et al. 2023), precipitation (e.g., Yue and Gebremichael 2020; Duda and Turner 2021; English et al. 2021), and atmospheric thermodynamic and kinematic profiles (e.g., Fovell and Gallagher 2020; MacDonald and Nowotarski 2023). Whereas these studies provide insights into model performance on seasonal and subseasonal time scales (e.g., Lee et al. 2019; Min et al. 2021), the key findings from these studies were reported irrespective of weather conditions.

Previous studies have also yet to address how model performance varies due to both gradual and rapid changes in near-surface atmospheric conditions caused by different land surface forcings (e.g., flash droughts, flash floods), airmass changes (e.g., during frontal passages and dryline passages), etc. We argue that model performance strongly depends on the atmospheric conditions under investigation, and model error diagnostics should be performed accordingly by classifying weather conditions without performing bulk statistics on model errors. Positive biases under one set of ambient atmospheric conditions and negative biases under another set of conditions, when averaged together, yield small mean biases. Consequently, these “global means” are not fully indicative of model performance and can obfuscate our ability to isolate the specific conditions in which the model biases are largest versus when model biases are smallest. In the example of the HRRR, which is the focus of the present study, knowledge of regime-specific model performance (i.e., MDMs as a function of different weather conditions) is anticipated to provide targeted areas for additional research and model development and help guide improvements to the land surface and atmospheric boundary layer (ABL) parameterizations in subsequent HRRR iterations, i.e., the Rapid Refresh Forecast System (RRFS; Dowell et al. 2022).

In this study, we use high-quality observations obtained from the U.S. Climate Reference Network (USCRN) over a 1-yr period (i.e., 1 January 2021–31 December 2021) to evaluate forecasts of air temperature, incoming shortwave radiation, and soil moisture from the HRRR over three subsets of different conditions (i.e., different near-surface heating rates, radiative regimes, and soil moisture regimes) to perform a weather-specific model evaluation, as summarized in Fig. 1. We then compare the results with model errors resulting from averaging across the entire range of atmospheric conditions observed during the 1-yr study period, which was previously reported in numerous previous studies, for example, L23 and which is the conventional model evaluation approach used to examine performance of operational NWP models.

Fig. 1.
Fig. 1.

(a) USCRN stations in CONUS (white squares). Stations that are omitted because of the water land-cover classification have an orange circle (cf. section 3). (b) Percentiles of dT/dt, SWd¯, and SM05¯ calculated across all USCRN stations for 2021 are shown. To show all variables on the same graph, dT/dt, SWd¯, and SM05¯ were multiplied by 10, 0.1, and 100, respectively. The black circles represent the 50th percentile; the error bars extend outward to the 25th and 75th percentiles; and the red circles are the 10th and 90th percentiles. (c) We summarize our model evaluation approach. Source of the background map in (a): GoogleEarth.

Citation: Weather and Forecasting 39, 5; 10.1175/WAF-D-23-0177.1

2. Datasets

a. USCRN

The USCRN currently comprises 114 stations installed at carefully selected sites in the contiguous United States (i.e., CONUS) and has been operating continuously across this domain since 2008 (Fig. 1a). The stations are spaced at the spatial resolution necessary to capture the temporal variability in temperature and precipitation trends over CONUS (Diamond et al. 2013). The spatial resolution of the USCRN stations over CONUS was determined using observations from thousands of stations in the Cooperative Observing Network (Vose and Menne 2004; Vose 2005; Diamond et al. 2013). At each USCRN station are in situ measurements of air temperature (T) at 1.5 m above ground level (AGL), surface skin temperature, precipitation, incoming shortwave radiation (SWd), and soil temperature and soil moisture (SM) at 5, 10, 20, 50, and 100 cm below the surface. Soil temperature and soil moisture are available at all five of these depths at 90 of the USCRN stations, whereas 5- and 10-cm measurements are available from 23 of the USCRN stations. Due to solid rock at the station at Torrey, Utah, there are no soil measurements available from that site (e.g., Bell et al. 2013).

The measurements from USCRN are of high quality; for example, T has an accuracy of ±0.3°C over the range from −50° to +50°C, and the raw T data have a resolution of 0.01°C (NOAA/NESDIS 2007). As this network has been well-described in previous work, we refer the reader to previous studies (e.g., Bell et al. 2013; Diamond et al. 2013; L23) that provide additional information about USCRN, including details about site selection, instrument type, data quality control and quality assurance procedures, etc. In the analyses in the present study, we focus on USCRN’s observations of T, SWd, and 5-cm SM.

b. HRRR

We used model output, available in GRIB2 format, from version 4 of the HRRR (hereinafter referred to as the HRRRv4) which has a 3-km grid spacing and 51 vertical layers that extend up to 15 hPa (e.g., Dowell et al. 2022). HRRRv4 uses the Rapid Update Cycle (RUC) land surface model (LSM) (e.g., Smirnova et al. 2016), Mellor–Yamada–Nakanishi–Niino eddy diffusivity mass flux (MYNN-EDMF) boundary layer scheme (Nakanishi and Niino 2004, 2009; Olson et al. 2019), MYNN surface-layer scheme (Olson et al. 2021), Thompson microphysics scheme (e.g., Thompson and Eidhammer 2014), and the Rapid Radiative Transfer Model Global for radiation (e.g., Iacono et al. 2008). We note, though, that the MYNN surface-layer scheme outputs 2-m T whereas USCRN measures T at 1.5 m AGL, and we do not explicitly account for these height discrepancies in our analyses. More details about the surface-layer scheme and other parameterization schemes used in the HRRR have been reported in, for example Benjamin et al. (2016), Dowell et al. (2022), and James et al. (2022).

3. Methods

The RUC LSM used in the HRRRv4 has 21 land-cover categories. Whereas 12 of these categories are represented at the USCRN stations, most of the USCRN stations have either the cropland or grassland land-cover type as was noted in L23. Eight stations have the water land-cover classification due to their proximity to lakes or oceans (Fig. 1a). Because the grid cells with the water land-cover classification have different characteristics (e.g., saturated soils) in the RUC LSM than the other land-cover classifications, we did not use the eight USCRN stations with the water land-cover classification in any analyses.

In this study we focus on the performance of the HRRRv4’s 18-h forecast to illustrate the different biases resulting from different near-surface atmospheric conditions. The 18-h forecast is run every hour; consequently, there are 8760 1-h time periods considered during the 1-yr study period at each of the USCRN stations. This forecast cycle is less sensitive to the model’s initial conditions and data assimilation than earlier forecast hours; therefore, evaluating the 18-h forecast is arguably a better assessment of the model physics than earlier forecast hours. However, using the same evaluations for earlier forecast hours (i.e., the 1-h forecast from the HRRRv4) yields similar results, indicating that our results are not strongly sensitive to our choice of forecast hour. We revisit this point in section 4.

We obtained HRRRv4 output from the NOAA Open Data Dissemination (NODD) program via the Amazon Web Services (AWS), and we used the procedure discussed in Blaylock et al. (2017). Following L23, we omitted observations of a given variable from a given station if >25% of the total expected observations within the experimental time frame were missing. For example, if Station X had 70% data completion over the period of record for 5-cm SM, but Station X had 80% data completion for T, we would omit the 5-cm SM observations from Station X from all further analyses, but we would still use the T observations from Station X. Incomplete data records are most common for 5-cm SM. In 2021, 70% of the USCRN stations have >75% data completion for 5-cm SM, whereas 99% of the USCRN stations have >75% data completion for T, and 98% of the USCRN stations have >75% data completion for SWd. The smaller SM data completion occurs because of soils freezing during the winter months at northern latitudes.

We then used the USCRN datasets to differentiate among meteorological regimes. In the present study we distinguish among ABL regimes using an encroachment model (e.g., Carson 1973; Tennekes 1973; Batchvarova and Gryning 1991). In encroachment models, the dynamics of turbulent entrainment are neglected, and the ABL growth rate is approximated as a function of the daytime heating rate (dT/dt) and ABL lapse rate. Ever since encroachment models were developed more than 50 years ago, they have successfully been used for example in recent studies on ABL growth and evolution (e.g., Pal et al. 2010, 2013). We computed dT/dt using the USCRN T observations to distinguish among days with large heating rates, and thus typically deeper ABLs, from days with small daytime heating rates and thus typically more shallow ABLs. We computed dT/dt as the T difference between 1400 and 0800 LST. Over the 1-yr study period, there was a large range in the observed daily values of dT/dt, as the daily dT/dt across all 114 USCRN stations in 2021 ranged from −2.5 to 4.8 K h−1. Cases in which T decreased during the daytime (i.e., days with dT/dt<0Kh−1) occurred on about 5% of all days and oftentimes had cold front passages that resulted in daytime T decreases. We performed sensitivity tests using different start and end times postsunrise and presunset, respectively, and found that our results remained independent of our choice for the starting hour or ending hour for computing dT/dt (not shown).

Additionally, we classified dT/dt into four different regimes based on histogram analyses of the percentiles, which we computed across all USCRN stations, to capture a range of magnitudes of near-surface forcings. We then determined the MDM by calculating the mean bias error (MBE) as the difference between the modeled and observed values for four dT/dt classifications:

  1. <25th percentile of dT/dt

  2. 25th–50th percentile of dT/dt

  3. 50th–75th percentile of dT/dt

  4. >75th percentile of dT/dt.

As shown in Fig. 1b, the 25th and 75th percentiles, computed across stations, for dT/dt were 0.57 and 1.40 K h−1, respectively. The largest dT/dt occurred over the semiarid U.S. Southwest and southern Great Plains where the 25th dT/dt percentile was ∼1 K h−1, and the 75th dT/dt percentile was >2 K h−1 (Figs. 2a–c). The smallest dT/dt generally occurred at coastal sites in the eastern United States. Once we computed dT/dt, we then analyzed the MBEs for the different dT/dt as a function of local standard time at each station. To facilitate a comparison with L23 (i.e., their Fig. 8), we report the results as a function of UTC rather than in local time. When considering the nighttime, i.e., roughly from 0200 to 1200 UTC, we used the dT/dt percentile for the upcoming daytime period. For the investigation of the spatial variability of the MBEs, we computed the MBE as a function of the difference between the HRRR and USCRN observations obtained between 1800 and 0000 UTC to represent the late afternoon mean values. Furthermore, we computed the means of the MBEs over the entire year to assess their spatial variability.

Fig. 2.
Fig. 2.

(a) 25th, (b) 50th, and (c) 75th percentiles of dT/dt at the USCRN stations. (d)–(f),(g)–(i) As in (a)–(c), but for the percentiles of SWd¯, and SM05¯, respectively. Note that not all stations are included in all plots due to the data availability for some variables (cf. section 3).

Citation: Weather and Forecasting 39, 5; 10.1175/WAF-D-23-0177.1

We repeated the analyses discussed above for different magnitudes of dT/dt, but used USCRN’s observations of mean daily shortwave radiation (SWd¯) and mean daily 5-cm soil moisture (SM05¯) to quantify the MDM across different radiative regimes and SM states, respectively, and compared these findings against the MDM over the entire study period. Over the study period, the 25th and 75th percentiles for SWd¯  (SM05¯) were 102 W m−2 (0.11 m3 m−3) and 250 W m−2 (0.31 m3 m−3), respectively (Figs. 1b,c). The SWd¯ and SM05¯ percentiles exhibited expected north–south and west–east gradients, respectively, with the largest SWd¯ and smallest SM05¯ occurring over the U.S. Southwest (Figs. 2d–i).

4. Results and discussion

a. Diurnal biases

1) Air temperature

When distinguishing among different heating rates, we find substantial differences in the mean diurnal cycle of T¯ MBE (Fig. 3a). For low heating rates (i.e., dT/dt < 25th percentile), the T¯ MBE is very small (∼0°C) during the nighttime but 2°C during the daytime which is largely in contrast to the subsets of days with larger heating rates as well as the mean across all cases (i.e., thick black line in Fig. 3). Days with the largest dT/dt (i.e., dT/dt > 75th percentile) have the largest T¯ MBE during the nighttime. On this subset of days, the MBE peaks ∼4°C at 1200 UTC but decreases during the daytime to around −1°C. These values are much different than the mean diurnal cycle in MBE computed across all days, whereby the HRRRv4 has a positive (i.e., warm) bias of ∼1°C during the nighttime and slightly negative (i.e., cool) bias between 1500 and 1800 UTC. Repeating these analyses using HRRR forecasts closer to the model’s initialization, i.e., using the 1-h HRRR forecast rather than the 18-h HRRR forecast, shows than the MBE amplitude is smaller in the 1-h HRRR forecast than in 18-h HRRR forecast, which is consistent with findings from L23 (their Fig. 8). However, the MBE for the different dT/dt percentiles follows the same diurnal pattern in the 1-h forecast as the 18-h forecast, suggesting that our results are not strongly sensitive to the choice of HRRR forecast hour that is evaluated.

Fig. 3.
Fig. 3.

Mean diurnal MBE cycle of (a) T¯, (b) SWd¯, and (c) SM05¯ for the different percentiles of dT/dt observed across all USCRN stations. (d)–(f),(g)–(i) As in (a)–(c), but for the different SWd¯ and SM05¯ percentiles, respectively. The purple, blue, orange, and red lines represent <25th, 25th–50th, 50th–75th, and >75th percentiles, respectively; the thick black line represents the mean across the entire study period.

Citation: Weather and Forecasting 39, 5; 10.1175/WAF-D-23-0177.1

Additionally, we find that varying dT/dt has substantial impacts on SWd¯ MBE. On days with the smallest dT/dt, the MBE reaches a daytime maximum of about 170 W m−2 (Fig. 3b). Conversely, days with the largest dT/dt yielded maximum MBEs of 100 W m−2. In contrast with these findings and, perhaps not surprisingly, the impacts of different dT/dt are more muted for the SM05¯ MBE (Fig. 3c). For all dT/dt percentiles, the MBE has a small mean negative bias.

These analyses highlight an important feature and a unique aspect of the MDM approach that we applied in the present study, which is that none of the relationships for the different scenarios follow the mean diurnal cycles. The mean diurnal cycle has been used in many previous studies evaluating the HRRR (e.g., Lee et al. 2019; Patel et al. 2021; He et al. 2023; L23). Our findings reveal the importance of a weather regime–specific approach for model evaluation and suggests that, when the MDM is not evaluated for different near-surface atmospheric conditions, the MDM bulk statistics (the MBE in this context) provide a somewhat dubious answer. Here, the MDM bulk statistics complicate our ability to properly isolate the scenarios in which model deficiencies are most substantial. In the case of T, the largest MBEs occur under the largest dT/dt. In these instances, the HRRR is likely not well representing land surface forcings during the early to mid-morning hours, which contributes to the large T biases. However, as noted above, the impacts of different observed dT/dt also affect the magnitude of the MBE for SWd¯. Furthermore, there are significant impacts on the SWd¯ MBE, with a larger positive MBE for small heating rates (Fig. 3c), and the MBE show a slightly mean negative bias for all dT/dt.

2) Incoming shortwave radiation

Previous studies have noted differences in the HRRR’s performance as a function of cloud cover (e.g., Min et al. 2021; L23). Accordingly, we specifically evaluate HRRRv4’s performance for different SWd¯ regimes (Figs. 3d–f). We find positive biases of nearly 200 W m−2 on days with the smallest SWd¯, and the HRRRv4 generally has a negative bias on days with the largest SWd¯. In contrast, the MBE resulting from averaging across all days was up to 100 W m−2 lower than the biases found on the cloudiest days. Furthermore, we note substantial impacts on the diurnal cycle of T¯ MBE. On days with the smallest SWd¯, there is a positive T¯ bias of about 3°C during the afternoon, and the MBE is about 2°C through the reminder of the day. In contrast, the HRRRv4 underestimates daytime T¯ by about 2°C during the daytime on days with the largest SWd¯ but, interestingly, slightly overestimates T¯ between about 0600 and 1200 UTC. As we found when evaluating the impact of dT/dt on the MBEs, different SWd¯ have a negligible impact on the SM05¯ MBE.

The larger positive biases on cloudy days (i.e., days with the smallest SWd) are known from previous studies evaluating the HRRR and have been attributed to the model’s difficulty resolving subgrid-scale clouds (e.g., Lee et al. 2019; Min et al. 2021; He et al. 2023). The MDM differences that arise as a function of varying SWd¯, coupled with findings from section 4a(1), underscore the need for caution when calculating MDM statistics for model evaluation without any constraints present.

3) Soil moisture

As for SWd¯, previous studies also noted differences in HRRR’s performance as a function of different soil moisture regimes. Over CONUS, the HRRRv4 overestimates SM05¯  for dry soils and underestimates SM05¯  for wet soils. These discrepancies have been attributed to a possible issue with soil moisture conductivity in the HRRR, which was speculated by L23. Because soil moisture helps govern the partitioning of available energy into sensible and latent heat (e.g., Stull 1988), it is critical to differentiate model performance among different soil moisture regimes to provide additional insights into model performance. Our analyses here indicate that, whereas SM MBE across all cases exhibit only a small negative bias, soil moisture biases exhibit substantial variability across the different SM05¯  regimes (Fig. 3i). There is a positive bias of ∼0.12 m3 m−3 when the observed SM05¯  is small (i.e., <25th percentile) and negative bias of 0.16 m3 m−3 when the observed SM05¯  is large (i.e., >75th percentile). This finding is consistent with L23 investigating the relationship between the SM05¯  MBE and observed SM05¯  over CONUS. The T¯ and SWd¯ MBEs, however, show little sensitivity to the SM regime (Figs. 3g,h); there is a slightly positive nighttime T¯ bias and slightly negative daytime T¯ bias which is consistent with the mean diurnal cycle across all days. Similarly, the SWd¯ MBE for all the different SM05¯ percentiles exhibit the same diurnal characteristics. The limited change in temperature may be related to the asymmetric responses of the near-surface atmosphere to soil moisture conditions. For example, Berg et al. (2014) noted that the ABL response to dry soil moisture conditions can lead to either warmer or cooler conditions, depending on the development of localized circulations (e.g., Stéfanon et al. 2014), clouds (e.g., Ek and Holtslag 2004; Lyons 2002), and precipitation (e.g., Findell and Eltahir 1997).

b. Spatial variability in biases

1) Air temperature

To further illustrate the impact of local changes in near-surface atmospheric conditions on HRRR biases, we consider the spatial variability among the USCRN sites in these biases and found substantial regional differences in MBEs (Fig. 4). The absolute values of the T¯ MBEs are considerably less over CONUS when all cases are averaged together (Fig. 4a) than when differentiating between days for two extreme dT/dt regimes, i.e., days with dT/dt < 25th percentile (Fig. 4b) versus days with dT/dt > 75th percentile (Fig. 4c). In the latter two instances, there are markedly negative and markedly positive, respectively, T biases across CONUS. Averaging these negative and positive biases together to determine the MDM spatial variability for all cases, as shown in Fig. 4a, greatly reduces the MDM magnitude which is not necessarily a true representation of model performance. This finding underscores the importance of partitioning MBE computation by daytime heating regimes across the CONUS to allow for a more comprehensive, weather-specific MDM experiment.

Fig. 4.
Fig. 4.

Spatial variability in (a) T¯ MBE across all percentiles, (b) T¯ MBE when dT/dt < 25th percentile, and (c) T¯ MBE when dT/dt > 75th percentile. (d) The spatial variability in SWd¯ MBE across all percentiles. (e),(f) The spatial variability in SWd¯ MBE when SWd¯<25thpercentile and SWd¯>75thpercentile, respectively. (g) The spatial variability in SM05¯ MBE across all percentiles. (h),(i) The spatial variability in SM05¯ MBE when SM05¯<25thpercentile and SM05¯>75thpercentile. Only values obtained between 1800 and 0000 UTC daily were used in these analyses.

Citation: Weather and Forecasting 39, 5; 10.1175/WAF-D-23-0177.1

Furthermore, when we evaluate the sensitivity of SWd¯ MBE and SM05¯ MBE to different percentiles of dT/dt, we find that the HRRR generally has larger SWd¯ MBE in the subset of cases with dT/dt < 25th percentile than for the cases with dT/dt > 75th percentile. However, the SM05¯ MBE does not greatly vary as a function of the different dT/dt percentiles (Fig. 5). In general, soil moisture controls land-atmosphere feedback processes, with higher (lower) soil moisture yielding lower (higher) near-surface temperatures (e.g., Ek and Holtslag 2004; Pal and Haeffelin 2015; Pal et al. 2020). However, our findings suggest that changes in dT/dt regimes occurring under various weather conditions do not directly impact soil moisture forecasts. Nevertheless, whether the chain of feedback processes related to weather conditions and associated precipitation regimes (i.e., rain rate, type, and droplet size distributions), and consequently the variability of soil moisture and temperature, are impacted by different dT/dt requires further investigation that is beyond the scope of the present study.

Fig. 5.
Fig. 5.

Spatial variability in (a) SWd¯ MBE when dT/dt < 25th percentile and (b) SWd¯ MBE when dT/dt > 75th percentile. (c),(d) The spatial variability in SM05¯ MBE when dT/dt < 25th percentile and dT/dt > 75th percentile, respectively. Only values obtained between 1800 and 0000 UTC daily were used in these analyses.

Citation: Weather and Forecasting 39, 5; 10.1175/WAF-D-23-0177.1

2) Incoming shortwave radiation

Consistent with the results reported in section 4a(2), there is a slightly positive MBE (Fig. 4d) across CONUS. The bias is largest on the cloudiest days, i.e., when SWd¯<25thpercentile (Fig. 4e) and exhibits little regional variability. In contrast and consistent with the mean diurnal MBE cycle (cf. Fig. 3e), the MBE for days with SWd¯>75th percentile is considerably smaller than days with the SWd¯<25th percentile (Fig. 4f). When we evaluate the sensitivity of T¯ MBE and SM05¯ MBE to different percentiles of SWd¯, we note considerable differences in the T¯ MBE for the different classifications of SWd¯; there is a much smaller T¯ MBE for low SWd¯ than for high SWd¯, with many locations having MBE > 3°C (Fig. 6). As is the case for different classes of dT/dt, SM05¯ MBE does not greatly vary as a function of different SWd¯.

Fig. 6.
Fig. 6.

Spatial variability in T¯ MBE when (a) SWd¯<25thpercentile and (b) SWd¯>75thpercentile. (c),(d) The spatial variability in SM05¯ MBE when SWd¯<25thpercentile and SWd¯>75thpercentile, respectively. Only values obtained between 1800 and 0000 UTC daily were used in these analyses.

Citation: Weather and Forecasting 39, 5; 10.1175/WAF-D-23-0177.1

3) Soil moisture

The SM05¯ MBE has a west-to-east gradient across all SM05¯ percentiles, with positive MBEs across the drier western United States and negative MBEs across the wetter eastern United States (Fig. 4g), in agreement with L23. When soils are dry (i.e., SM05¯<25thpercentile), the MBEs are typically positive across much of CONUS (Fig. 4h), whereas when soils are wet (i.e., SM05¯>75thpercentile) MBEs are typically negative, with the exception of the very dry stations in the southwestern United States (Fig. 4i). These findings provide further evidence to support our hypothesis. In this particular scenario, contrasting MDMs under two different SM states (cf. Figs. 4h,i) are averaged out and thus are not represented in the bulk statistics (cf. Fig. 4g).

Finally, when we evaluate the sensitivity of T¯ MBE and SWd¯ MBE to different percentiles of SM05¯, we find that, when dry conditions are present (i.e., SM05¯<25thpercentile), there are strong negative T¯ MBEs (i.e., <−3°C) in the Rockies and positive MBEs across the eastern United States. The strongly negative T¯ MBEs in the Rockies are also present for wet conditions (i.e., SM05¯>75thpercentile), whereas the T¯ MBEs are much smaller (Fig. 7). In contrast, there is little impact of the different SM05¯ classifications on SWd¯ MBE.

Fig. 7.
Fig. 7.

(a) Spatial variability in T¯ MBE when SM05¯<25thpercentile and (b) SM05¯>75thpercentile. (c),(d) The spatial variability in SWd¯ MBE when SM05¯<25thpercentile and SM05¯>75thpercentile, respectively. Only values obtained between 1800 and 0000 UTC daily were used in these analyses.

Citation: Weather and Forecasting 39, 5; 10.1175/WAF-D-23-0177.1

5. Summary, conclusions, and outlook

We demonstrated that weather regime–specific evaluations of NWP models are required to better distinguish cases when the models perform well versus when models perform poorly. We used one year of observations obtained from the USCRN coupled with HRRRv4 output and found substantial differences in model performance for the different near-surface atmospheric conditions. When averaged across all near-surface atmospheric conditions that we observed during the 1-yr study period, which is the typical approach used for evaluating weather forecasting models, the model biases tend to be rather small. However, the model biases greatly varied when distinguishing among different near-surface heating rates and radiative regimes. We found a negative T MBE for small heating rates and positive T MBE for larger heating rates, as well as substantial differences in T and SWd MBE as a function of different radiative regimes.

Information about the aforementioned biases in operational forecasting models such as the HRRR and conducting additional similar evaluations of model performance is important for isolating the scenarios when model deficiencies are largest. The approach is arguably better than evaluating model performance using bulk statistics computed across a large range of different near-surface atmospheric conditions. Having knowledge about the conditional behavior of model biases provides insights into processes that are not well represented within the modeling system and are being pursued using a process-oriented verification approach described by, for example Turner et al. (2020). Doing so will permit future opportunities to evaluate NWP models under a myriad of other types of meteorological conditions. These conditions include, for example, nonsheared versus sheared ABLs (e.g., Fedorovich and Conzemius 2008), dry versus moist ABLs (e.g., Stull 1988), warm versus cold sectors of the ABL before and after frontal passages (e.g., Pal et al. 2021; Clark et al. 2022), etc. The use of a process-oriented verification approach will allow for targeted areas for improving the land surface and boundary layer parameterization schemes used within operational NWP models, like the HRRR and its successors (i.e., the RRFS). Finally, a regime-specific model evaluation, like the one developed in the present study, may similarly be used to help evaluate the efficacy of other geophysical models, including for example inverse carbon transport models, general circulation models, etc.

Acknowledgments.

We thank NOAA/Air Resources Laboratory’s engineers for maintaining USCRN and our colleagues from NOAA/National Centers for Environmental Information for ensuring a high-quality dataset from USCRN. SP was partially supported by the NOAA Grant NA21OAR4590361. We also thank the two anonymous reviewers whose suggestions helped us to improve the technical and scientific content of the manuscript. Finally, we note that the results and conclusions of this study and views expressed herein are those of the authors and may not necessarily reflect the views of NOAA or the Department of Commerce.

Data availability statement.

The USCRN datasets are archived by NOAA’s National Centers for Environmental Information and can be accessed at https://www.ncei.noaa.gov/access/crn/qcdatasets.html. The HRRRv4 GRIB2 forecast output is available from the NOAA Open Data Dissemination program via Amazon Web Services’ archive, i.e. https://registry.opendata.aws/noaa-hrrr-pds/.

REFERENCES

  • Batchvarova, E., and S.-E. Gryning, 1991: Applied model for the growth of the daytime mixed layer. Bound.-Layer Meteor., 56, 261–274, https://doi.org/10.1007/BF00120423.

    • Search Google Scholar
    • Export Citation

  • Bell, J. E., and Coauthors, 2013: U.S. climate reference network soil moisture and temperature observations. J. Hydrometeor., 14, 977–988, https://doi.org/10.1175/JHM-D-12-0146.1.

    • Search Google Scholar
    • Export Citation

  • Benjamin, S. G., and Coauthors, 2016: A North American hourly assimilation and model forecast cycle: The Rapid Refresh. Mon. Wea. Rev., 144, 1669–1694, https://doi.org/10.1175/MWR-D-15-0242.1.

    • Search Google Scholar
    • Export Citation

  • Berg, A., B. R. Lintner, K. L. Findell, S. Malyshev, P. C. Loikith, and P. Gentine, 2014: Impact of soil moisture–atmosphere interactions on surface temperature distribution. J. Climate, 27, 7976–7993, https://doi.org/10.1175/JCLI-D-13-00591.1.

    • Search Google Scholar
    • Export Citation

  • Blaylock, B. K., J. D. Horel, and S. T. Liston, 2017: Cloud archiving and data mining of high-resolution rapid refresh forecast model output. Comput. Geosci., 109, 43–50, https://doi.org/10.1016/j.cageo.2017.08.005.

    • Search Google Scholar
    • Export Citation

  • Carson, D. J., 1973: The development of a dry inversion‐capped convectively unstable boundary layer. Quart. J. Roy. Meteor. Soc., 99, 450–467, https://doi.org/10.1002/qj.49709942105.

    • Search Google Scholar
    • Export Citation

  • Clark, N. E., S. Pal, and T. R. Lee, 2022: Empirical evidence for the frontal modification of atmospheric boundary layer depth variability over land. J. Appl. Meteor. Climatol., 61, 1041–1063, https://doi.org/10.1175/JAMC-D-21-0099.1.

    • Search Google Scholar
    • Export Citation

  • Diamond, H. J., and Coauthors, 2013: U.S. climate reference network after one decade of operations: Status and assessment. Bull. Amer. Meteor. Soc., 94, 485–498, https://doi.org/10.1175/BAMS-D-12-00170.1.

    • Search Google Scholar
    • Export Citation

  • Dowell, D. C., and Coauthors, 2022: The High-Resolution Rapid Refresh (HRRR): An hourly updating convection-allowing forecast model. Part 1: Motivation and system description. Wea. Forecasting, 37, 1371–1395, https://doi.org/10.1175/WAF-D-21-0151.1.

    • Search Google Scholar
    • Export Citation

  • Duda, J. D., and D. D. Turner, 2021: Large-sample application of radar reflectivity object-based verification to evaluate HRRR warm-season forecasts. Wea. Forecasting, 36, 805–821, https://doi.org/10.1175/WAF-D-20-0203.1.

    • Search Google Scholar
    • Export Citation

  • Ek, M. B., and A. A. M. Holtslag, 2004: Influence of soil moisture on boundary layer cloud development. J. Hydrometeor., 5, 86–99, https://doi.org/10.1175/1525-7541(2004)005<0086:IOSMOB>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • English, J. M., D. D. Turner, T. I. Alcott, W. R. Moninger, J. L. Bytheway, R. Cifelli, and M. Marquis, 2021: Evaluating operational and experimental HRRR model forecasts of atmospheric river events in California. Wea. Forecasting, 36, 1925–1944, https://doi.org/10.1175/WAF-D-21-0081.1.

    • Search Google Scholar
    • Export Citation

  • Fedorovich, E., and R. Conzemius, 2008: Effects of wind shear on the atmospheric convective boundary layer structure and evolution. Acta Geophys., 56, 114–141, https://doi.org/10.2478/s11600-007-0040-4.

    • Search Google Scholar
    • Export Citation

  • Findell, K. L., and E. A. B. Eltahir, 1997: An analysis of the soil moisture-rainfall feedback, based on direct observations from Illinois. Water Resour. Res., 33, 725–735, https://doi.org/10.1029/96WR03756.

    • Search Google Scholar
    • Export Citation

  • Fovell, R. G., and A. Gallagher, 2020: Boundary layer and surface verification of the High-Resolution Rapid Refresh, version 3. Wea. Forecasting, 35, 2255–2278, https://doi.org/10.1175/WAF-D-20-0101.1.

    • Search Google Scholar
    • Export Citation

  • He, S., D. D. Turner, S. G. Benjamin, J. B. Olson, T. G. Smirnova, and T. Meyers, 2023: Evaluation of the near-surface variables in the HRRR weather model using observations from the ARM SGP site. J. Appl. Meteor. Climatol., 62, 769–780, https://doi.org/10.1175/JAMC-D-23-0003.1.

    • Search Google Scholar
    • Export Citation

  • Iacono, M. J., J. S. Delamere, E. J. Mlawer, M. W. Shephard, S. A. Clough, and W. D. Collins, 2008: Radiative forcing by longlived greenhouse gases: Calculations with the AER radiative transfer models. J. Geophys. Res., 113, D13103, https://doi.org/10.1029/2008JD009944.

    • Search Google Scholar
    • Export Citation

  • James, E. P., and Coauthors, 2022: The High-Resolution Rapid Refresh (HRRR): An hourly updating convection-allowing forecast model. Part II: Forecast performance. Wea. Forecasting, 37, 1397–1417, https://doi.org/10.1175/WAF-D-21-0130.1.

    • Search Google Scholar
    • Export Citation

  • Lee, T. R., M. Buban, D. D. Turner, T. P. Meyers, and C. B. Baker, 2019: Evaluation of the High-Resolution Rapid Refresh (HRRR) model using near-surface meteorological and flux observations from northern Alabama. Wea. Forecasting, 34, 635–663, https://doi.org/10.1175/WAF-D-18-0184.1.

    • Search Google Scholar
    • Export Citation

  • Lee, T. R., R. D. Leeper, T. Wilson, H. J. Diamond, T. P. Meyers, and D. D. Turner, 2023: Using the U.S. climate reference network to identify biases in near-and sub-surface meteorological fields in the High-Resolution Rapid Refresh (HRRR) weather prediction model. Wea. Forecasting, 38, 879–900, https://doi.org/10.1175/WAF-D-22-0213.1.

    • Search Google Scholar
    • Export Citation

  • Lyons, T., 2002: Clouds prefer native vegetation. Meteor. Atmos. Phys., 80, 131–140, https://doi.org/10.1007/s007030200020.

  • MacDonald, L. M., and C. J. Nowotarski, 2023: Verification of Rapid Refresh and High-Resolution Rapid Refresh model variables in tornadic tropical cyclones. Wea. Forecasting, 38, 655–675, https://doi.org/10.1175/WAF-D-22-0117.1.

    • Search Google Scholar
    • Export Citation

  • Min, L., D. R. Fitzjarrald, Y. Du, B. E. J. Rose, J. Hong, and Q. Min, 2021: Exploring sources of surface bias in HRRR using New York State Mesonet. J. Geophys. Res. Atmos., 126, e2021JD034989, https://doi.org/10.1029/2021JD034989.

    • Search Google Scholar
    • Export Citation

  • Nakanishi, M., and H. Niino, 2004: An improved Mellor-Yamada level-3 model with condensation physics: Its design and verification. Bound.-Layer Meteor., 112, 1–31, https://doi.org/10.1023/B:BOUN.0000020164.04146.98.

    • Search Google Scholar
    • Export Citation

  • Nakanishi, M., and H. Niino, 2009: Development of an improved turbulence closure model for the atmospheric boundary layer. J. Meteor. Soc. Japan, 87, 895–912, https://doi.org/10.2151/jmsj.87.895.

    • Search Google Scholar
    • Export Citation

  • NOAA/NESDIS, 2007: United States Climate Reference Network (USCRN) functional requirements document. NOAA Doc. NOAA-CRN/OSD-2003-0009R1UD0, 27 pp., https://www.ncei.noaa.gov/pub/data/uscrn/documentation/program/X040_d0.pdf.

  • Olson, J. B., J. S. Kenyon, W. M. Angevine, J. M. Brown, M. Pagowski, and K. Suselj, 2019: A description of the MYNN-EDMF scheme and the coupling to other components in WRF-ARW. NOAA Tech. Memo. OAR GSD-61, 42 pp., https://repository.library.noaa.gov/view/noaa/19837.

  • Olson, J. B., T. Smirnova, J. S. Kenyon, D. D. Turner, J. M. Brown, W. Zheng, and B. W. Green, 2021: A description of the MYNN surface-layer scheme. NOAA Tech. Memo. OAR GSL-67, 26 pp., https://doi.org/10.25923/f6a8-bc75.

  • Pal, S., and M. Haeffelin, 2015: Forcing mechanisms governing diurnal, seasonal, and interannual variability in the boundary layer depths: Five years of continuous lidar observations over a suburban site near Paris. J. Geophys. Res. Atmos., 120, 11 936–11 956, https://doi.org/10.1002/2015JD023268.

    • Search Google Scholar
    • Export Citation

  • Pal, S., A. Behrendt, and V. Wulfmeyer, 2010: Elastic-backscatter-lidar-based characterization of the convective boundary layer and investigation of related statistics. Ann. Geophys., 28, 825–847, https://doi.org/10.5194/angeo-28-825-2010.

    • Search Google Scholar
    • Export Citation

  • Pal, S., M. Haeffelin, and E. Batchvarova, 2013: Exploring a geophysical process‐based attribution technique for the determination of the atmospheric boundary layer depth using aerosol lidar and near‐surface meteorological measurements. J. Geophys. Res. Atmos., 118, 9277–9295, https://doi.org/10.1002/jgrd.50710.

    • Search Google Scholar
    • Export Citation

  • Pal, S., T. R. Lee, and N. E. Clark, 2020: The 2019 Mississippi and Missouri River flooding and its impact on atmospheric boundary layer dynamics. Geophy. Res. Lett., 47, e2019GL086933, https://doi.org/10.1029/2019GL086933.

    • Search Google Scholar
    • Export Citation

  • Pal, S., N. E. Clark, T. R. Lee, M. Conder, and M. Buban, 2021: When and where horizontal advection is critical to alter atmospheric boundary layer dynamics over land: The need for a conceptual framework. Atmos. Res., 264, 105825, https://doi.org/10.1016/j.atmosres.2021.105825.

    • Search Google Scholar
    • Export Citation

  • Patel, R. N., S. E. Yuter, M. A. Miller, S. R. Rhodes, L. Bain, and T. W. Peele, 2021: The diurnal cycle of winter season temperature errors in the operational global forecast system (GFS). Geophys. Res. Lett., 48, e2021GL095101, https://doi.org/10.1029/2021GL095101.

    • Search Google Scholar
    • Export Citation

  • Smirnova, T. G., J. M. Brown, S. G. Benjamin, and J. S. Kenyon, 2016: Modifications to the Rapid Update Cycle Land Surface Model (RUC LSM) available in the Weather Research and Forecasting (WRF) Model. Mon. Wea. Rev., 144, 1851–1865, https://doi.org/10.1175/MWR-D-15-0198.1.

    • Search Google Scholar
    • Export Citation

  • Stéfanon, M., P. Drobinski, F. D’Andrea, C. Lebeaupin-Brossier, and S. Bastin, 2014: Soil moisture-temperature feedbacks at meso-scale during summer heat waves over western Europe. Climate Dyn., 42, 1309–1324, https://doi.org/10.1007/s00382-013-1794-9.

    • Search Google Scholar
    • Export Citation

  • Stull, R. B., 1988: An Introduction to Boundary Layer Meteorology. Kluwer Academic, 666 pp.

  • Tennekes, H., 1973: A model for the dynamics of the inversion above a convective boundary layer. J. Atmos. Sci., 30, 558–567, https://doi.org/10.1175/1520-0469(1973)030<0558:AMFTDO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Thompson, G., and T. Eidhammer, 2014: A study of aerosol impacts on clouds and precipitation development in a large winter cyclone. J. Atmos. Sci., 71, 3636–3658, https://doi.org/10.1175/JAS-D-13-0305.1.

    • Search Google Scholar
    • Export Citation

  • Turner, D. D., and Coauthors, 2020: A verification approach used in developing the Rapid Refresh and other numerical weather prediction models. J. Oper. Meteor., 8, 39–53, https://doi.org/10.15191/nwajom.2020.0803.

    • Search Google Scholar
    • Export Citation

  • Vose, R. S., 2005: Reference station networks for monitoring climatic change in the conterminous United States. J. Climate, 18, 5390–5395, https://doi.org/10.1175/JCLI3600.1.

    • Search Google Scholar
    • Export Citation

  • Vose, R. S., and M. J. Menne, 2004: A method to determine station density requirements for climate observing networks. J. Climate, 17, 2961–2971, https://doi.org/10.1175/1520-0442(2004)017<2961:AMTDSD>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Yue, H., and M. Gebremichael, 2020: Evaluation of High-Resolution Rapid Refresh (HRRR) forecasts for extreme precipitation. Environ. Res. Commun., 2, 065004, https://doi.org/10.1088/2515-7620/ab9002.

    • Search Google Scholar
    • Export Citation
Save
Cover Weather and Forecasting
Weather and Forecasting

  • Batchvarova, E., and S.-E. Gryning, 1991: Applied model for the growth of the daytime mixed layer. Bound.-Layer Meteor., 56, 261–274, https://doi.org/10.1007/BF00120423.

    • Search Google Scholar
    • Export Citation

  • Bell, J. E., and Coauthors, 2013: U.S. climate reference network soil moisture and temperature observations. J. Hydrometeor., 14, 977–988, https://doi.org/10.1175/JHM-D-12-0146.1.

    • Search Google Scholar
    • Export Citation

  • Benjamin, S. G., and Coauthors, 2016: A North American hourly assimilation and model forecast cycle: The Rapid Refresh. Mon. Wea. Rev., 144, 1669–1694, https://doi.org/10.1175/MWR-D-15-0242.1.

    • Search Google Scholar
    • Export Citation

  • Berg, A., B. R. Lintner, K. L. Findell, S. Malyshev, P. C. Loikith, and P. Gentine, 2014: Impact of soil moisture–atmosphere interactions on surface temperature distribution. J. Climate, 27, 7976–7993, https://doi.org/10.1175/JCLI-D-13-00591.1.

    • Search Google Scholar
    • Export Citation

  • Blaylock, B. K., J. D. Horel, and S. T. Liston, 2017: Cloud archiving and data mining of high-resolution rapid refresh forecast model output. Comput. Geosci., 109, 43–50, https://doi.org/10.1016/j.cageo.2017.08.005.

    • Search Google Scholar
    • Export Citation

  • Carson, D. J., 1973: The development of a dry inversion‐capped convectively unstable boundary layer. Quart. J. Roy. Meteor. Soc., 99, 450–467, https://doi.org/10.1002/qj.49709942105.

    • Search Google Scholar
    • Export Citation

  • Clark, N. E., S. Pal, and T. R. Lee, 2022: Empirical evidence for the frontal modification of atmospheric boundary layer depth variability over land. J. Appl. Meteor. Climatol., 61, 1041–1063, https://doi.org/10.1175/JAMC-D-21-0099.1.

    • Search Google Scholar
    • Export Citation

  • Diamond, H. J., and Coauthors, 2013: U.S. climate reference network after one decade of operations: Status and assessment. Bull. Amer. Meteor. Soc., 94, 485–498, https://doi.org/10.1175/BAMS-D-12-00170.1.

    • Search Google Scholar
    • Export Citation

  • Dowell, D. C., and Coauthors, 2022: The High-Resolution Rapid Refresh (HRRR): An hourly updating convection-allowing forecast model. Part 1: Motivation and system description. Wea. Forecasting, 37, 1371–1395, https://doi.org/10.1175/WAF-D-21-0151.1.

    • Search Google Scholar
    • Export Citation

  • Duda, J. D., and D. D. Turner, 2021: Large-sample application of radar reflectivity object-based verification to evaluate HRRR warm-season forecasts. Wea. Forecasting, 36, 805–821, https://doi.org/10.1175/WAF-D-20-0203.1.

    • Search Google Scholar
    • Export Citation

  • Ek, M. B., and A. A. M. Holtslag, 2004: Influence of soil moisture on boundary layer cloud development. J. Hydrometeor., 5, 86–99, https://doi.org/10.1175/1525-7541(2004)005<0086:IOSMOB>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • English, J. M., D. D. Turner, T. I. Alcott, W. R. Moninger, J. L. Bytheway, R. Cifelli, and M. Marquis, 2021: Evaluating operational and experimental HRRR model forecasts of atmospheric river events in California. Wea. Forecasting, 36, 1925–1944, https://doi.org/10.1175/WAF-D-21-0081.1.

    • Search Google Scholar
    • Export Citation

  • Fedorovich, E., and R. Conzemius, 2008: Effects of wind shear on the atmospheric convective boundary layer structure and evolution. Acta Geophys., 56, 114–141, https://doi.org/10.2478/s11600-007-0040-4.

    • Search Google Scholar
    • Export Citation

  • Findell, K. L., and E. A. B. Eltahir, 1997: An analysis of the soil moisture-rainfall feedback, based on direct observations from Illinois. Water Resour. Res., 33, 725–735, https://doi.org/10.1029/96WR03756.

    • Search Google Scholar
    • Export Citation

  • Fovell, R. G., and A. Gallagher, 2020: Boundary layer and surface verification of the High-Resolution Rapid Refresh, version 3. Wea. Forecasting, 35, 2255–2278, https://doi.org/10.1175/WAF-D-20-0101.1.

    • Search Google Scholar
    • Export Citation

  • He, S., D. D. Turner, S. G. Benjamin, J. B. Olson, T. G. Smirnova, and T. Meyers, 2023: Evaluation of the near-surface variables in the HRRR weather model using observations from the ARM SGP site. J. Appl. Meteor. Climatol., 62, 769–780, https://doi.org/10.1175/JAMC-D-23-0003.1.

    • Search Google Scholar
    • Export Citation

  • Iacono, M. J., J. S. Delamere, E. J. Mlawer, M. W. Shephard, S. A. Clough, and W. D. Collins, 2008: Radiative forcing by longlived greenhouse gases: Calculations with the AER radiative transfer models. J. Geophys. Res., 113, D13103, https://doi.org/10.1029/2008JD009944.

    • Search Google Scholar
    • Export Citation

  • James, E. P., and Coauthors, 2022: The High-Resolution Rapid Refresh (HRRR): An hourly updating convection-allowing forecast model. Part II: Forecast performance. Wea. Forecasting, 37, 1397–1417, https://doi.org/10.1175/WAF-D-21-0130.1.

    • Search Google Scholar
    • Export Citation

  • Lee, T. R., M. Buban, D. D. Turner, T. P. Meyers, and C. B. Baker, 2019: Evaluation of the High-Resolution Rapid Refresh (HRRR) model using near-surface meteorological and flux observations from northern Alabama. Wea. Forecasting, 34, 635–663, https://doi.org/10.1175/WAF-D-18-0184.1.

    • Search Google Scholar
    • Export Citation

  • Lee, T. R., R. D. Leeper, T. Wilson, H. J. Diamond, T. P. Meyers, and D. D. Turner, 2023: Using the U.S. climate reference network to identify biases in near-and sub-surface meteorological fields in the High-Resolution Rapid Refresh (HRRR) weather prediction model. Wea. Forecasting, 38, 879–900, https://doi.org/10.1175/WAF-D-22-0213.1.

    • Search Google Scholar
    • Export Citation

  • Lyons, T., 2002: Clouds prefer native vegetation. Meteor. Atmos. Phys., 80, 131–140, https://doi.org/10.1007/s007030200020.

  • MacDonald, L. M., and C. J. Nowotarski, 2023: Verification of Rapid Refresh and High-Resolution Rapid Refresh model variables in tornadic tropical cyclones. Wea. Forecasting, 38, 655–675, https://doi.org/10.1175/WAF-D-22-0117.1.

    • Search Google Scholar
    • Export Citation

  • Min, L., D. R. Fitzjarrald, Y. Du, B. E. J. Rose, J. Hong, and Q. Min, 2021: Exploring sources of surface bias in HRRR using New York State Mesonet. J. Geophys. Res. Atmos., 126, e2021JD034989, https://doi.org/10.1029/2021JD034989.

    • Search Google Scholar
    • Export Citation

  • Nakanishi, M., and H. Niino, 2004: An improved Mellor-Yamada level-3 model with condensation physics: Its design and verification. Bound.-Layer Meteor., 112, 1–31, https://doi.org/10.1023/B:BOUN.0000020164.04146.98.

    • Search Google Scholar
    • Export Citation

  • Nakanishi, M., and H. Niino, 2009: Development of an improved turbulence closure model for the atmospheric boundary layer. J. Meteor. Soc. Japan, 87, 895–912, https://doi.org/10.2151/jmsj.87.895.

    • Search Google Scholar
    • Export Citation

  • NOAA/NESDIS, 2007: United States Climate Reference Network (USCRN) functional requirements document. NOAA Doc. NOAA-CRN/OSD-2003-0009R1UD0, 27 pp., https://www.ncei.noaa.gov/pub/data/uscrn/documentation/program/X040_d0.pdf.

  • Olson, J. B., J. S. Kenyon, W. M. Angevine, J. M. Brown, M. Pagowski, and K. Suselj, 2019: A description of the MYNN-EDMF scheme and the coupling to other components in WRF-ARW. NOAA Tech. Memo. OAR GSD-61, 42 pp., https://repository.library.noaa.gov/view/noaa/19837.

  • Olson, J. B., T. Smirnova, J. S. Kenyon, D. D. Turner, J. M. Brown, W. Zheng, and B. W. Green, 2021: A description of the MYNN surface-layer scheme. NOAA Tech. Memo. OAR GSL-67, 26 pp., https://doi.org/10.25923/f6a8-bc75.

  • Pal, S., and M. Haeffelin, 2015: Forcing mechanisms governing diurnal, seasonal, and interannual variability in the boundary layer depths: Five years of continuous lidar observations over a suburban site near Paris. J. Geophys. Res. Atmos., 120, 11 936–11 956, https://doi.org/10.1002/2015JD023268.

    • Search Google Scholar
    • Export Citation

  • Pal, S., A. Behrendt, and V. Wulfmeyer, 2010: Elastic-backscatter-lidar-based characterization of the convective boundary layer and investigation of related statistics. Ann. Geophys., 28, 825–847, https://doi.org/10.5194/angeo-28-825-2010.

    • Search Google Scholar
    • Export Citation

  • Pal, S., M. Haeffelin, and E. Batchvarova, 2013: Exploring a geophysical process‐based attribution technique for the determination of the atmospheric boundary layer depth using aerosol lidar and near‐surface meteorological measurements. J. Geophys. Res. Atmos., 118, 9277–9295, https://doi.org/10.1002/jgrd.50710.

    • Search Google Scholar
    • Export Citation

  • Pal, S., T. R. Lee, and N. E. Clark, 2020: The 2019 Mississippi and Missouri River flooding and its impact on atmospheric boundary layer dynamics. Geophy. Res. Lett., 47, e2019GL086933, https://doi.org/10.1029/2019GL086933.

    • Search Google Scholar
    • Export Citation

  • Pal, S., N. E. Clark, T. R. Lee, M. Conder, and M. Buban, 2021: When and where horizontal advection is critical to alter atmospheric boundary layer dynamics over land: The need for a conceptual framework. Atmos. Res., 264, 105825, https://doi.org/10.1016/j.atmosres.2021.105825.

    • Search Google Scholar
    • Export Citation

  • Patel, R. N., S. E. Yuter, M. A. Miller, S. R. Rhodes, L. Bain, and T. W. Peele, 2021: The diurnal cycle of winter season temperature errors in the operational global forecast system (GFS). Geophys. Res. Lett., 48, e2021GL095101, https://doi.org/10.1029/2021GL095101.

    • Search Google Scholar
    • Export Citation

  • Smirnova, T. G., J. M. Brown, S. G. Benjamin, and J. S. Kenyon, 2016: Modifications to the Rapid Update Cycle Land Surface Model (RUC LSM) available in the Weather Research and Forecasting (WRF) Model. Mon. Wea. Rev., 144, 1851–1865, https://doi.org/10.1175/MWR-D-15-0198.1.

    • Search Google Scholar
    • Export Citation

  • Stéfanon, M., P. Drobinski, F. D’Andrea, C. Lebeaupin-Brossier, and S. Bastin, 2014: Soil moisture-temperature feedbacks at meso-scale during summer heat waves over western Europe. Climate Dyn., 42, 1309–1324, https://doi.org/10.1007/s00382-013-1794-9.

    • Search Google Scholar
    • Export Citation

  • Stull, R. B., 1988: An Introduction to Boundary Layer Meteorology. Kluwer Academic, 666 pp.

  • Tennekes, H., 1973: A model for the dynamics of the inversion above a convective boundary layer. J. Atmos. Sci., 30, 558–567, https://doi.org/10.1175/1520-0469(1973)030<0558:AMFTDO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Thompson, G., and T. Eidhammer, 2014: A study of aerosol impacts on clouds and precipitation development in a large winter cyclone. J. Atmos. Sci., 71, 3636–3658, https://doi.org/10.1175/JAS-D-13-0305.1.

    • Search Google Scholar
    • Export Citation

  • Turner, D. D., and Coauthors, 2020: A verification approach used in developing the Rapid Refresh and other numerical weather prediction models. J. Oper. Meteor., 8, 39–53, https://doi.org/10.15191/nwajom.2020.0803.

    • Search Google Scholar
    • Export Citation

  • Vose, R. S., 2005: Reference station networks for monitoring climatic change in the conterminous United States. J. Climate, 18, 5390–5395, https://doi.org/10.1175/JCLI3600.1.

    • Search Google Scholar
    • Export Citation

  • Vose, R. S., and M. J. Menne, 2004: A method to determine station density requirements for climate observing networks. J. Climate, 17, 2961–2971, https://doi.org/10.1175/1520-0442(2004)017<2961:AMTDSD>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Yue, H., and M. Gebremichael, 2020: Evaluation of High-Resolution Rapid Refresh (HRRR) forecasts for extreme precipitation. Environ. Res. Commun., 2, 065004, https://doi.org/10.1088/2515-7620/ab9002.

    • Search Google Scholar
    • Export Citation
  • <sc>Fig</sc>. 1.
    View in gallery
    Fig. 1.

    (a) USCRN stations in CONUS (white squares). Stations that are omitted because of the water land-cover classification have an orange circle (cf. section 3). (b) Percentiles of dT/dt, SWd¯, and SM05¯ calculated across all USCRN stations for 2021 are shown. To show all variables on the same graph, dT/dt, SWd¯, and SM05¯ were multiplied by 10, 0.1, and 100, respectively. The black circles represent the 50th percentile; the error bars extend outward to the 25th and 75th percentiles; and the red circles are the 10th and 90th percentiles. (c) We summarize our model evaluation approach. Source of the background map in (a): GoogleEarth.

  • <sc>Fig</sc>. 2.
    View in gallery
    Fig. 2.

    (a) 25th, (b) 50th, and (c) 75th percentiles of dT/dt at the USCRN stations. (d)–(f),(g)–(i) As in (a)–(c), but for the percentiles of SWd¯, and SM05¯, respectively. Note that not all stations are included in all plots due to the data availability for some variables (cf. section 3).

  • <sc>Fig</sc>. 3.
    View in gallery
    Fig. 3.

    Mean diurnal MBE cycle of (a) T¯, (b) SWd¯, and (c) SM05¯ for the different percentiles of dT/dt observed across all USCRN stations. (d)–(f),(g)–(i) As in (a)–(c), but for the different SWd¯ and SM05¯ percentiles, respectively. The purple, blue, orange, and red lines represent <25th, 25th–50th, 50th–75th, and >75th percentiles, respectively; the thick black line represents the mean across the entire study period.

  • <sc>Fig</sc>. 4.
    View in gallery
    Fig. 4.

    Spatial variability in (a) T¯ MBE across all percentiles, (b) T¯ MBE when dT/dt < 25th percentile, and (c) T¯ MBE when dT/dt > 75th percentile. (d) The spatial variability in SWd¯ MBE across all percentiles. (e),(f) The spatial variability in SWd¯ MBE when SWd¯<25thpercentile and SWd¯>75thpercentile, respectively. (g) The spatial variability in SM05¯ MBE across all percentiles. (h),(i) The spatial variability in SM05¯ MBE when SM05¯<25thpercentile and SM05¯>75thpercentile. Only values obtained between 1800 and 0000 UTC daily were used in these analyses.

  • <sc>Fig</sc>. 5.
    View in gallery
    Fig. 5.

    Spatial variability in (a) SWd¯ MBE when dT/dt < 25th percentile and (b) SWd¯ MBE when dT/dt > 75th percentile. (c),(d) The spatial variability in SM05¯ MBE when dT/dt < 25th percentile and dT/dt > 75th percentile, respectively. Only values obtained between 1800 and 0000 UTC daily were used in these analyses.

  • <sc>Fig</sc>. 6.
    View in gallery
    Fig. 6.

    Spatial variability in T¯ MBE when (a) SWd¯<25thpercentile and (b) SWd¯>75thpercentile. (c),(d) The spatial variability in SM05¯ MBE when SWd¯<25thpercentile and SWd¯>75thpercentile, respectively. Only values obtained between 1800 and 0000 UTC daily were used in these analyses.

  • <sc>Fig</sc>. 7.
    View in gallery
    Fig. 7.

    (a) Spatial variability in T¯ MBE when SM05¯<25thpercentile and (b) SM05¯>75thpercentile. (c),(d) The spatial variability in SWd¯ MBE when SM05¯<25thpercentile and SM05¯>75thpercentile, respectively. Only values obtained between 1800 and 0000 UTC daily were used in these analyses.

Fig. 1.

(a) USCRN stations in CONUS (white squares). Stations that are omitted because of the water land-cover classification have an orange circle (cf. section 3). (b) Percentiles of dT/dt, SWd¯, and SM05¯ calculated across all USCRN stations for 2021 are shown. To show all variables on the same graph, dT/dt, SWd¯, and SM05¯ were multiplied by 10, 0.1, and 100, respectively. The black circles represent the 50th percentile; the error bars extend outward to the 25th and 75th percentiles; and the red circles are the 10th and 90th percentiles. (c) We summarize our model evaluation approach. Source of the background map in (a): GoogleEarth.
Fig. 2.

(a) 25th, (b) 50th, and (c) 75th percentiles of dT/dt at the USCRN stations. (d)–(f),(g)–(i) As in (a)–(c), but for the percentiles of SWd¯, and SM05¯, respectively. Note that not all stations are included in all plots due to the data availability for some variables (cf. section 3).
Fig. 3.

Mean diurnal MBE cycle of (a) T¯, (b) SWd¯, and (c) SM05¯ for the different percentiles of dT/dt observed across all USCRN stations. (d)–(f),(g)–(i) As in (a)–(c), but for the different SWd¯ and SM05¯ percentiles, respectively. The purple, blue, orange, and red lines represent <25th, 25th–50th, 50th–75th, and >75th percentiles, respectively; the thick black line represents the mean across the entire study period.
Fig. 4.

Spatial variability in (a) T¯ MBE across all percentiles, (b) T¯ MBE when dT/dt < 25th percentile, and (c) T¯ MBE when dT/dt > 75th percentile. (d) The spatial variability in SWd¯ MBE across all percentiles. (e),(f) The spatial variability in SWd¯ MBE when SWd¯<25thpercentile and SWd¯>75thpercentile, respectively. (g) The spatial variability in SM05¯ MBE across all percentiles. (h),(i) The spatial variability in SM05¯ MBE when SM05¯<25thpercentile and SM05¯>75thpercentile. Only values obtained between 1800 and 0000 UTC daily were used in these analyses.
Fig. 5.

Spatial variability in (a) SWd¯ MBE when dT/dt < 25th percentile and (b) SWd¯ MBE when dT/dt > 75th percentile. (c),(d) The spatial variability in SM05¯ MBE when dT/dt < 25th percentile and dT/dt > 75th percentile, respectively. Only values obtained between 1800 and 0000 UTC daily were used in these analyses.
Fig. 6.

Spatial variability in T¯ MBE when (a) SWd¯<25thpercentile and (b) SWd¯>75thpercentile. (c),(d) The spatial variability in SM05¯ MBE when SWd¯<25thpercentile and SWd¯>75thpercentile, respectively. Only values obtained between 1800 and 0000 UTC daily were used in these analyses.
Fig. 7.

(a) Spatial variability in T¯ MBE when SM05¯<25thpercentile and (b) SM05¯>75thpercentile. (c),(d) The spatial variability in SWd¯ MBE when SM05¯<25thpercentile and SM05¯>75thpercentile, respectively. Only values obtained between 1800 and 0000 UTC daily were used in these analyses.
All Time Past Year Past 30 Days
Abstract Views 101 101 0
Full Text Views 1573 1573 361
PDF Downloads 520 520 30