1. Introduction
Clouds, particularly stratus and stratocumulus clouds, which globally cover an average of 23% of ocean area and 12% of land area in the annual mean (Wood 2012), strongly modulate the radiative balance at Earth’s surface and alter sensible and latent heat (moisture) fluxes between Earth’s surface and atmosphere. These changes in fluxes, in turn, modulate development of the planetary boundary layer (PBL), in particular, affecting the temperature and moisture in the soil and near-surface atmosphere. Short-range (1–12 h) forecasts are thus often particularly dependent upon the accuracy of initial cloud fields. The model spinup problem (e.g., Donner and Rasch 1989; Colle et al. 1999), evident in a delay in the development of clouds and precipitation in the early stages of a forecast without initial cloud data, is a challenge faced by cold-start or noncycled models producing short-range forecasts of aviation-sensitive weather parameters and high-impact convective weather.
Atmospheric models can be initialized without any initial cloud information (initial cloud and ice mixing ratios assumed to be zero) and will still generate cloud fields within ~2–8 h. Inaccurate radiation (shortwave, primarily) can, however, lead to excessive temperatures during daytime and faulty evolution of the PBL including for the prethunderstorm environment (e.g., Benjamin and Carlson 1986). Inaccurate nighttime initialization of clouds can lead to large errors in nighttime cooling near the surface in conditions of light winds through errant downward longwave radiative fluxes. Furthermore, because ceiling and visibility hazards account for 27% of general aviation fatalities over the last 30 years (Fultz and Ashley 2016), accurate clouds and precipitation forecasts are critical. Therefore, skillful prediction of PBL evolution through accurate specification of initial 3D hydrometeor fields is particularly important for accurate short-range cloud, precipitation and convective storm forecasts used in a wide variety of decision-making including that for aviation and energy applications.
Cloud observations are routinely derived from satellites and by surface-based ceilometers and visibility sensors—these provide possible input for operational weather forecast models. Previous nowcasting techniques using satellite cloud data without any prior background cloud information included were described by Albers et al. (1996), Thompson et al. (1997), Häggmark et al. (2000), and Bayler et al. (2000). A cloud initialization method with satellite and background data were developed by de Haan and van der Veen (2014) for the European High-Resolution Limited Area Model (HIRLAM). Macpherson et al. (1996) describes an earlier cloud assimilation used in the Met Office mesoscale model modifying model forecasts with proxy relative humidity profiles created from surface-based cloud, satellite, and radar observational data. Explicit assimilation of cloud fraction in models at the Met Office via 4DVar is described by Renshaw and Francis (2011), documenting improvements to 6-h cloud cover and 2-m temperature forecasts when the vertical spreading of pseudohumidity observations is limited.
Cloudiness data from satellites and surface instruments are used here in a data assimilation (DA) method to update 3D prognostic hydrometeor fields. We broadly define “data assimilation” here to include data fusion to accommodate “edge information,” such as cloud cover, which is not easily treated by variational DA (e.g., Asadi et al. 2019 for nonfractional ice cover). The cloud/hydrometeor assimilation method described herein 1) updates the prior 3D hydrometeor fields from a previous 1h forecast via clearing (setting to zero) and building, 2) modifies water vapor mixing ratio and temperature to be consistent with cloud building and clearing to enable retention, and 3) applies interconsistency checks between satellite, radar, and METAR cloud data and the background surface (land/snow/water/ice).
We call this a stratiform cloud-hydrometeor (SCH) DA technique since it is for stratiform clouds only (convective clouds treated through radar DA; Weygandt et al. 2021, manuscript submitted to Wea. Forecasting) and since it directly updates prognostic cloud hydrometeor fields. It has been applied and refined in NOAA hourly updated models as an option within the NOAA GSI DA system (Kleist et al. 2009) and applied to the NOAA 3-km High-Resolution Rapid Refresh (HRRR; Dowell et al. 2021, manuscript submitted to Mon. Wea. Rev.; James et al. 2021, manuscript submitted to Mon. Wea. Rev.) and 13-km Rapid Refresh (RAP) models (Benjamin et al. 2016, hereafter B16) including HRRRv4 and RAPv5 implemented at NCEP on 2 December 2020. Both HRRR and RAP use the Advanced Research version of Weather Research and Forecasting (WRF-ARW) model (Skamarock et al. 2008). The SCHDA technique used in RAP/HRRR complements a hybrid ensemble-variational DA also within GSI (Hu et al. 2017; B16) for hourly assimilation of water vapor, temperature, and wind observations (B16; Dowell et al. 2021, manuscript submitted to Mon. Wea. Rev.; James et al. 2020). SCHDA also complements in RAP/HRRR the assimilation of radar reflectivity (and lightning) using latent heating for the RAP within digital filter initialization (DFI; Peckham et al. 2016; outlined in B16). In B16, specific attention is given to improvements of short-range ceiling forecasts from hourly DA. An earlier, less effective version of this cloud DA technique was applied to the hourly updated NOAA Rapid Update Cycle model starting in 2002 (Benjamin et al. 2004a,b). The cloud-assimilation technique described herein is separate from but can complement more widely used assimilation of satellite-observed radiances in cloudy areas (cloudy radiances) in global models (McNally 2009; Bauer et al. 2010; Okamoto et al. 2014; Migliorini and Candy 2019; Zhu et al. 2019).
2. Cloud-hydrometeor-related observations and model first guess
Cloud observation information both at analysis time and at prior times (via recent model forecasts) affect forecast fields through the SCHDA technique. The total cloud information content includes having observations of where the cloud is (e.g., its base or top) and where it is not (regions of the sky that are cloud-free). We determine this cloud/no-cloud information from both satellite- or surface-based observations. Fractional cloud information may be available from these observations. A summary of both observational and model background datasets used in the SCHDA are summarized in Table 1.
3D prognostic variables updated and observations used in stratiform cloud/hydrometeor (SCH) assimilation. [Updated from Tables 4 (obs) and 5 (prognostic variables) from B16.]
a. Cloud observations
This section describes the observational datasets of cloud coverage used in the SCHDA scheme.
1) METAR—Cloud and visibility
The aviation routine weather report (METAR) data are typically generated at least once an hour, often from an Automated Surface Observing System (ASOS) located at airports, military bases, and other sites, and sometimes from augmented observations or from trained observers or forecasters. A regular METAR contains a report of near-surface atmospheric elements including 10-m wind, 2-m temperature, 2-m dewpoint, and barometric pressure together with weather information such as precipitation type or obscuration (fog or mist) and trend, cloud height and cover, and visibility.1
METARs can include information on as many as six layers of the cloud-base height and cloud amount based on automated lidar-based ceilometer (O’Connor et al. 2004; Kollias et al. 2004; Pal et al. 1992) or manual trained-observer data. The cloud amount can be described using five categories: clear (set as a cloud fraction of 0.0), few (set as 0.1 as midpoint of the 0.01–0.2 interval in Fig. 1), scattered (set as 0.3), broken (0.7), and overcast (1.0). A “clear” METAR cloud observation is an indicator of no clouds in the column above the surface location, at least up to 12 000 ft (3650 m) above ground level (AGL; O’Connor et al. 2004).2
Cloud-ceiling observations from METARs from 1200 UTC 25 Jan 2017 over the Rapid Refresh (RAP) domain mapped to (a) RAP model level k = 1 (near surface), (b) RAP model level k = 6 (350–400 m AGL), and (c) maximum cloud cover at any level. (Stations shown here with 100-km circles).
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
An example of METAR cloud cover information over the RAP domain, which includes a large portion of the Northern Hemisphere including North America, is shown in Fig. 1 for a typical winter day. Other METAR-reported information used in the cloud/hydrometeor assimilation includes the precipitation category, current weather, and visibility. Visibility is generally measured by attenuation from a laser-based runway-visual-range sensor (RVR; e.g., Boudala et al. 2012). If an obscuration from fog or mist is reported at the same METAR station, the visibility is converted to a specific cloud (or ice) mixing ratio using an inversion of the relationship described by Stoelinga and Warner (1999, their Table 1).
2) Satellite
Satellites are the second major observing platform for the SCHDA. They provide a comprehensive horizontal view of cloud systems on a scale not possible by other means, especially for remote areas including over oceans. An infrared image from GOES-East (Fig. 2) over much of the United States for the same time as for Fig. 1 indicates cloud cover over much of the northern CONUS and in the lower Mississippi Valley with much more precise horizontal cloud cover than available with METAR data. The cloud-top temperature, cloud-top pressure, and cloud fraction are derived using GOES-East and GOES-West infrared and visible imager data by the NASA Langley Satellite ClOud and Radiation Property retrieval System (SatCORPS) based on methods described by Minnis et al. (2008, 2020) and Trepte et al. (2019). NESDIS uses similar methods as described in Foster and Heidinger (2014). In examples shown here (Fig. 3 over North America, Fig. 4 for CONUS), only GOES cloud retrieval fields are used (2-km resolution); similar data from polar-orbiting satellite sensors such as MODIS can also be used. Cloud fraction (Fig. 3c) is determined within a given area (~12 km) using the number of observed cloudy pixels divided by the total number of pixels.
GOES infrared (3.9-μm band) image valid at 1215 UTC 25 Jan 2017. Image provided by the Cooperative Institute for Research in the Atmosphere (CIRA).
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
Cloud-top fields derived from GOES, also for 1215 UTC 25 Jan 2017, showing (a) cloud-top temperature (K), (b) cloud-top pressure (hPa), and (c) cloud fraction over the RAP domain after data processing. The data processing adds clear areas [white in (b)] into cloud-top pressure field but not for the cloud-top temperature in (a), where white represents missing values that include areas outside the observation domain and some clear areas.
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
As in Fig. 3, but for a smaller domain and showing only (a) cloud-top temperature (K) and (b) cloud-top pressure (hPa). Vertical cross-section location for later figures is shown in red in (b).
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
3) Radar and lightning observations
Radar and lightning observations give information related to precipitating hydrometeors (snow, rain, graupel) but not directly about cloud water or cloud-ice hydrometeors. These observations are used to drive the latent heating–radar–DFI technique described briefly in B16 and also in Weygandt and Benjamin (2007), Weygandt et al. (2008a,b), and Weygandt et al. (2021, manuscript submitted to Wea. Forecasting). however, these observations are not part of the stratiform-specific SCHDA described here. These observation types are included in the observation consistency checks (Table 2, more information on these checks in section 3a).
Data consistency issues and related observation quality-control (QC) checks performed in SCHDA prior to the use of the observations in the assimilation procedure. Tskin is the skin temperature, bkg is the background data from the previous forecast at the nearest grid point, and FOVs are satellite-based fields-of-view. See section 3 for more information.
b. Model background
This section describes the background fields used in the SCH assimilation, both directly (e.g., 3D hydrometeor forecasts themselves) and indirectly (e.g., allowing inference on the phase and vertical location of cloud hydrometeors).
Prior information from short-range model forecasts provides important information on the hydrometeor fields for the analysis. With the Thompson and Eidhammer (2014) microphysics scheme used in the RAP and HRRR models, prognostic 3D hydrometeor fields available for this analysis include mixing ratios of cloud water (qc), rainwater (qr), cloud ice (qi), snow (qs), and graupel (qg). Other prognostic fields treated in the SCH assimilation include number concentrations per unit volume for rain droplets (Nr), cloud droplets (Nc), and ice particles (Ni; Table 1). The prior forecast 3D subgrid-scale (SGS) cloud fraction (Olson et al. 2019a,b) is also used in an SCHDA variant to prevent cloud building under partial cloudy conditions as an experiment (Table 3).
RAP cycled experiments regarding stratiform cloud-hydrometeor (SCH) assimilation. Key differences are in boldface. Experiments were carried out for a summer period (15–25 Jul 2018) and a winter period (1–11 Feb 2019).
From these explicit hydrometeor fields (including SGS cloud fraction), variables important for aviation operations, ceiling and visibility, are diagnosed as described in Benjamin et al. (2020). Ceiling is diagnosed here from the closest to surface level of qc or qi exceeding a threshold and also from boundary layer-top RH. An alternative ceiling diagnostic using subgrid cloud fraction (found best using 45% in summer) is now being tested but not used in this paper. Visibility is calculated from the magnitude of hydrometeor mixing ratios (qc, qi, qr, qs) at the lowest 2 prognostic levels of the model (also see original algorithm from Stoelinga and Warner (1999)).
Also from the model background, skin temperature from the top level of the land surface model and from the water representation where appropriate (e.g., sea surface temperature, lake-surface temperature) is used for comparison with satellite brightness temperatures as part of the QC procedures described in section 3a and Table 2.
Finally, water vapor and temperature background fields also contribute to the SCH assimilation and are modified as described below.
3. Components and application of SCH assimilation
This section describes the methods of SCHDA in three principal areas: (i) QC of observations, (ii) merging into 3D observed cloud/clear fields, and (iii) merging with model background fields.
a. Quality control (QC) of observations
Before modifying 3D hydrometeor and other 3D model fields as components of this cloud assimilation technique, procedures are undertaken to ensure quality of the input observations. These QC procedures are summarized in Table 2 and detailed below.
METAR, radar, and satellite observations usually agree for observing cloudy and clear conditions. However, occasionally, instrument and/or representativeness errors can lead to discrepancies among the data sources. Geostationary satellite observations are considered most reliable (because these observations come from a single platform with monitoring by NASA and NOAA) where available in these consistency checks, compared to METAR cloud and radar data (as these datasets are comprised from many different instruments, each of which could have its own characteristics). For example, if radar data show an echo where geostationary data indicate no clouds, the radar echo data are flagged to a do-not-use status (Table 2, test 1). Similarly, lightning stroke data are flagged when found in a cloud-free area according to satellite data at approximately the same time window (currently set within 15 min of analysis time).
The model background field aids in QC of the observations, especially for satellite observations. The background skin and cloud-top temperatures are compared at each grid point, flagging points where the observed cloud-top data are too close to the background skin temperature. This check resolves ambiguity caused by snow cover and for other causes of very stable layers near the surface (Table 2, test 2). A related “warm stratus” allowance in the QC procedure allows satellite-indicated cloud-top regions to be used when the satellite cloud-top temperature is sufficiently warmer than the skin temperature to remove ambiguity (more likely in polar regions, Table 2, test 3). For marine clouds, cloud-top pressure is recalculated using the GOES-retrieved cloud-top temperature with the local model-specific background temperature/humidity profile to obtain a new estimate of cloud-top pressure as consistent as possible with the background boundary layer structure (Table 2, test 4). The profiles of cloud amount and height are checked to ensure that these values are vertically consistent (e.g., the cloud level above an overcast layer must have a missing value – Table 2, test 5). Finally, since this assimilation technique is limited to stratiform clouds, no cloud building is performed if the background local lapse rate exceeds a threshold given by
b. Generation of a 3D cloud-information field
A special challenge of the SCHDA is identifying the best-possible forward model (to provide an observation-background difference of a common variable) enabled by mapping cloud/clear information onto the 3D model background grid, noting that cloud observations provide only the fraction (which could be zero), whereas the prognostic cloud variables in the analysis are in the form of 3D mixing ratios and number concentrations. The first step of SCHDA is to generate a 3D cloud-information (cloud fraction) array (designated “3D-cloudinfo” in this paper) based on METAR information and high-resolution satellite-observed cloud/clear information. These 3D-cloudinfo data are defined on the 3D model background grid. The 3D cloud-information process assigns to each 3D grid volume, based on each observation type, one of these three status indicators: cloudy, clear, or unknown. Both METAR and satellite observations allow estimation of a cloud fraction (from 0.0 to 1.0), so cloud fraction is part of this cloud-information field. The preliminary separate cloud-information fields for satellite and METAR data are then merged into a combined observation-based 3D cloud-information field.
The SCHDA components are described using the same case study from 1200 UTC 25 Jan 2017 shown in Figs. 1 and 2. Vertical cross sections of the partial and full cloud-information fields (Fig. 5, southern U.S. location in Fig. 4b) reflect satellite cloud coverage in the western part of this cross section mostly in the higher troposphere, roughly corresponding to model levels 18–24 (Fig. 5a). No information can be inferred through this cloud layer, so beneath this layer, for the preliminary satellite-only contribution, 3d-cloudinfo is assigned as unknown. METAR low-ceiling observations in this area (limited to 12 000 ft AGL as described earlier) were in the lower troposphere, resulting in partial 3d-cloudinfo data (Fig. 5b) and finally, the combined 3d-cloudinfo data (Fig. 5c). Application of 3d-cloudinfo to qc and qi for this same location (Fig. 6) is discussed in section 3c(3)(i). Observation inconsistencies are resolved toward satellite data (cf. Figs. 5b and 5c for x = 35–37) as described in Table 2 (check 1). Other details on this mapping are described in the subsections below. This 3d-cloudinfo field in Fig. 5c is typical: horizontally detailed cloud-top information from satellite and intermittent cloud-base information from METARs, dependent on METAR station density.
Vertical cross sections of components of the 3D cloud-information array (3d-cloudinfo), showing (a) with satellite only, (b) with METAR only, and (c) for all data, for 1200 UTC 25 Jan 2017 and for the Texas–Florida section shown by the red line in Fig. 4b. Vertical axis is for RAP model levels (sigma spacing in B16, their Table 7). Full horizontal axis is ~1040 km. The horizontal dashed lines indicate approximate height of tropopause.
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
Vertical cross sections of cloud-ice (qi; g kg−1, above dotted line for this location) and cloud water (qc; g kg−1, below dotted line) before and after applications of the cloud-information array (cloudinfo). (a) Background, (b) analysis with satellite cloud top only, (c) analysis with METAR cloud observations only, and (d) analysis with both satellite and METAR cloud obs. Plots are for 1200 UTC 25 Jan 2017 and for the Texas–Florida section shown by the red line in Fig. 4b. Horizontal dashed lines indicate approximate height of tropopause.
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
1) Mapping of METAR data onto 3D cloud-observation information field
Since METAR observations only provide the heights of cloud base and cloud amounts in each cloud layer, the SCH assimilation requires an estimate of the horizontal scale and depth of the cloud layers for assignment to the 3d-cloudinfo array (Fig. 5b).
For projection of the METAR cloud profile onto the model vertical profile, a “cloudy” value is assigned to the closest vertical model level below the observed cloud base in a given profile. This assignment allows best approximation to retaining aviation flight rules thresholds for ceiling [e.g., Instrument Flight Rules (IFR) at 1000 ft (~0.30 km) above ground level].
Currently, all METAR-observed cloud (in absence of moist unstable conditions–test 6 in Table 2) is assumed to be stratiform cloud with 300-m depth in the absence of precipitation [close to the mean stratocumulus depth estimated from liquid-water path data (Wood 2012, p. 2378)], or allowed to be up to 1000-m depth with precipitation in the column shown by radar observations or present in the surface observation. This depth is used to form a cloud layer above each cloud base level to generate a coverage profile. The vertical profile currently used in the HRRR and RAP models has spacing of about 60 m starting at level 3 up to about 400–500 m in the middle troposphere (B16, Table 7).
Cloud fraction is assigned in the 3d-cloudinfo profile as 0 for all clear points, 0.3 for scattered, 0.7 for broken, and 1.0 for overcast. “Cloudy” values (nonzero cloud fraction) are assigned in the 3d-cloudinfo array for least two model layers to estimate minimum stratiform cloud thickness and also to increase the chance of cloud retention. These cloud layers from METAR data are indicated in blue (fraction > 0.6: BKN, OVC) or green (0.01–0.6: FEW, SCT) color in Fig. 5b. Visibility observations are also treated as cloud-present observations for the lowest two layers in 3d-cloudinfo in the model if fog is reported in the METAR “current weather” information.
The horizontal representativeness of the METAR cloud observations is assumed to be up to 100 km (rough estimate of minimum horizontal extent of stratiform cloud sheets). Detailed satellite cloud imagery and the background cloud field are both applied to refine (or confirm) this METAR-related cloud-building/-clearing area to a much smaller (or same) region, as explained in section 3c.
2) Mapping of satellite data onto 3D cloud-observation information field
Satellite-based cloud fraction is defined as 0.0 down to the cloud-top pressure level interpolated to the nearest vertical model level (Fig. 5a). If the cloud-top temperature at a given grid point passes the consistency checks in Table 2, then 3d-cloudinfo will be set to the observed cloud-fraction (percentage of cloud pixels within grid area) with a depth of at least two model layers. The local ceiling/hydrometeor horizontal details from the 1-h background forecast are retained unless satellite cloud coverage indicates a conflicting cloud pattern.
c. Application of 3D cloud information data to clearing and building
After establishing the 3D cloud information field, a data fusion procedure is then applied using this observed 3D field with the background 3D hydrometeor (and water vapor and temperature) fields. The prognostic variables to be updated in the assimilation are summarized in Table 1 and were described in section 2b. Background values of these variables are taken from the forecast (1-h forecasts for the RAP or HRRR models). The final 3D cloud/precipitation hydrometeor fields are combinations of retrieved cloud/precipitation, background cloud/precipitation, and 3D cloud-coverage information and 3D radar reflectivity information.
The SCHDA process is sequential, first clearing nonzero background cloud hydrometeors where the 3D-cloudinfo indicate clear conditions, and then building hydrometeors in forecast clear 3D volumes where the observations (via 3D-cloudinfo) indicate cloud presence. If the 3D grid point is classified as “unknown” in the 3D-cloudinfo data, the background cloud/precipitation values are retained as the final results of the analysis. For instance, local cloud details in the background forecast are retained if the METAR observation agrees with the background nearest-gridpoint cloudy or clear conditions. To the extent that the background contains orographic or other local effects on cloud patterns which agree with METAR observations, these pre-existing horizontal cloud patterns will be retained, although 3d-cloudinfo may reflect a vertical displacement to the cloud layer based on the METAR ceiling observation.
At this point, no introduction of the observed cloud fraction (available in 3D-cloudinfo) is applied even though this is planned in the future by modifying the 3D SGS cloud fraction in turbulence schemes such as the MYNN scheme (Olson et al. 2019a,b) used in HRRR/RAP models. The SCHDA cloud-building is omitted in a separate experiment described in section 4 when SGS cloud fraction < 0.45 even if qc and qi are zero.
1) Modifications to temperature and water vapor for cloud building and clearing
Modifications to the thermodynamic environment (temperature and water vapor) need to occur in the SCH assimilation to support the cloud hydrometeor changes. These modifications are based on physical understanding of clouds and their thermodynamic environments. Cloud layers at the top of the boundary layer are accompanied by warmer potential temperatures above, consistent with the vertical profile of mixing in the typical boundary layer (e.g., Wyngaard 1985). Elevated mixed layers (Carlson et al. 1983) also exhibit nearly isentropic conditions capped by another inversion, sometimes accompanied by cloud layers at those levels (Corfidi et al. 2008). To match this structure, the SCH assimilation introduces saturation (building) or subsaturation (clearing) in a 3D grid volume by conserving virtual potential temperature (θυ) with changes to both water vapor and temperature (Table 1, columns 3 and 4). This θυ-conservation treatment is a refinement over cloud-assimilation changes only to water vapor (e.g., de Haan and van der Veen 2014).
In applying θυ conservation in an example of cloud building (Fig. 7b) based on a nearby METAR ceiling observation, a multilayer cloud is introduced around 920–930 hPa. Saturation is introduced by cooling the layer and adding water vapor while conserving θυ, a unique solution determined iteratively. The structure is roughly consistent with that seen in observed soundings in environments with an elevated mixed layer and boundary layer stratiform clouds. Similarly, subsaturation to 80% RH [with respect to water or ice dependent on temperature; see 3c(3)(i)] is introduced for 3D grid volumes with cloud clearing via both drying and warming solved by conserving θυ. This effect is appropriate for stable clouds and avoids introducing convective instability.
Effect on vertical profiles of temperature and water vapor (using skew T–logp display) (left) without and (right) with cloud/hydrometeor assimilation. For Victoria, TX, for analysis at 0000 UTC 21 May 2019 using HRRR analysis. See section 3c(1) for application of θυ conservation for both cloud building and clearing.
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
2) Clearing
Clearings of precipitating and nonprecipitating hydrometeors are treated separately. Again, Table 1 (last column) gives a brief overview on clearing and the zero-cloud volume (Fig. 5c) from the observation-based 3d-cloudinfo used as the basis for this clearing. Cloud assignment from 3d-cloudinfo is currently applied with “binary cloud” for ≥ 0.6 and “clear” for 3d-cloudinfo < 0.6 (admittedly too high a threshold value for clearing and another parameter that should be refined).
Clearing of nonprecipitating hydrometeors consists of changing to zero the values for qc, qi, Nc, and Ni, as evident in comparing Figs. 6a and 6d for upper-level clouds for points 1–20 and for low-level clouds for points 35–37. In 3D volumes where cloud clearing is applied, subsaturation is applied to the local 3D-gridpoint water vapor mixing ratio and temperature values as described in section 3c(1).
In clearing precipitating hydrometeors, any nonzero mixing ratios (qc, qi) and corresponding number concentrations in any 3D grid volume with 3d-cloudinfo = 0 are simply set to zero for rain (qr), snow (qs), and graupel (qg) and no subsaturation is applied. This may occur over any geographic area with satellite coverage. Three-dimensional radar data which shows an observed 3D volume with no echo is also used to clear 3D precipitating hydrometeors (usually only over land). No subsaturation is applied for these 3D volumes with clearing of precipitating hydrometeors.
3) Building
Most 3D grid volumes are forecast in the background to be cloud-free [zero cloud water (qc) or ice mixing ratio (qi)]. But some number of these 3D forecast-clear volumes can be indicated as “cloudy” volumes by the observations in 3D-cloudinfo, defined here with cloud fraction > 0.6. The subsequent cloud-building in such 3D grid volumes is described below. An example of low-level cloud building is evident in Fig. 6d for grid points 65–68. Some cloud cover in the HRRR-RAP model is represented by SGS cloud fraction (appendix B in B16; Olson et al. 2019b), a variable considered in an SCHDA option discussed later.
(i) Building of nonprecipitating cloud hydrometeors
At each 3D cloudy volume (as summarized in the 3d-cloudinfo field) without cloud in the background, qc (or qi) is specified as the minimum of 5% (0.5%) of saturation vapor pressure (with respect to water or ice dependent on temperature) and 0.05 g kg−1 (0.02 g kg−1). These values are far below the typical autoconversion values (from cloud to rain) at 0.2–0.7 g kg−1 dependent on local Nc and Ni, (G. Thompson 2021, personal communication). Regarding phase for hydrometeor building, qi is specified when the 3D gridpoint temperature is less than 263 K, qc when the temperature is greater than 268 K, and qc–qi mixed phase using linear interpolation between 263–268 K (found to be appropriate for the Thompson–Eidhammer microphysics). The cloud microphysics scheme quickly adapts from this estimate to its own equilibrium state related to its internal equations but with relatively little additional latent heating or cooling. Changes to qc and qi for the 25 Jan 2017 vertical cross section are shown in Fig. 6 (cf. Fig. 6a vs Fig. 6d), with qc change found in model levels 1–10 (lowest ~150 hPa) and with building/clearing for qi at the higher model levels 20–28 (upper troposphere).
When clouds are built, the cloud number concentrations (Nc and Ni) are specified at relatively high values within the range found in the Thompson–Eidhammer scheme (108 m−3 for Nc and 106 m−3 for Ni). For model grid points near to a visibility observation indicating fog, qc–qi values are set with more precision by reversing the visibility diagnostic (Stoelinga and Warner 1999, their Table 1; Benjamin et al. 2020, their section 2.G.iv). For the HRRR/RAP application of the SCHDA, no building of qc–qi is performed above 1200 m AGL (just above the aviation-sensitive 3000-ft ceiling threshold for instrument-free flight operations) to avoid introduction of an excessive moist bias in water vapor and RH, but qc–qi building was allowed at all levels for the case study in Figs. 3–6.
(ii) Building of precipitation hydrometeors
Precipitation hydrometeors (qr or qs) based on radar reflectivity (Z) data only are added to the final analysis in two conditions—one for relatively cold near-surface conditions (i.e., if the 2-m temperature is less than 5°C) and a separate very limited method for low reflectivity for warmer conditions for 2-m temperature (i.e., allowing building light rain or drizzle only for these warmer surface conditions). For these hydrometeors, qr and Nr are assigned using the Marshall and Palmer (1948) relationship, and qs is assigned based on Thompson et al. (2008). No temperature or water vapor modification is applied.
d. Examples for application of SCH assimilation
Most of the cloud information available for the SCHDA is from the prior forecasts carrying effects of previous cloud observations and physical consistency with the 3D dynamical and thermodynamical processes in the forecast model. Examples are shown here from both the 13-km RAP model (Fig. 8) with changes consistent with satellite data in Fig. 3 and the 3-km HRRR model (Figs. 9, 10).
Cloud-top height (in thousands of feet, kft) for (a) background and (b) analysis also for 13-km RAP for 1215 UTC 25 Jan 2017 (1000 ft ≈ 0.3 km).
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
Cloud-top fields (in thousands of feet, kft) for (a) HRRR 1-h forecast background, (b) HRRR analysis, (c) HRRR analysis increment (analysis minus background), and (d) HRRR 1-h forecast of cloud-cover fraction (including subgrid cloud), valid at 1600 UTC 12 Sep 2020.
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
Ceiling fields for (a) HRRR 1-h forecast background, (b) HRRR analysis, and (c) HRRR analysis increment (analysis minus background for changes > 1 kft).
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
Analysis increment fields from SCHDA are small in horizontal area relative to overall cloud fields. For the winter case (25 Jan 2017) described earlier, cloud-top changes for the 13-km RAP are evident only in small details (<~80 km) for lower clouds over land areas and adjacent oceanic areas and in some reduction in upper-level clouds over the same areas. In a daytime example from September 2020 using the smaller-domain 3-km HRRR, cloud-top (Fig. 9) structure was dominated by an upper-level low over the U.S. Midwest with midday low clouds from MN to OK and over the U.S. East Coast area from PA to AL. The cloud-top increment field (Fig. 9c) from the SCHDA indicated primarily cloud building at this time and few areas of cloud clearing. The upper-tropospheric clouds were already well-forecast in the 1-h prediction (Fig. 9a) near Lakes Michigan and Superior and for the tropical storm over the Florida Keys. Cloud-radiative effects in HRRR and RAP are represented by both explicit clouds (i.e., nonzero qc, qi) and through SGS clouds as discussed earlier. For this case, the cloud-cover fraction including these SGS clouds is also presented from the same 1-h HRRR forecast in Fig. 9d. In general, most of the large-scale cloud information is accurately represented in the 1-h forecast background, and the increment fields (Fig. 9c for cloud-top and Fig. 10c for ceiling) provide details to the 3D cloud structure, similar qualitatively to the smaller-scale differences from comparing Figs. 6a and 6d for a previous case. (Some changes to low-level clouds are not shown in Figs. 9c, 10c due to its 1000-ft contour interval.)
Some of the cloud building (Fig. 9c, Fig. 10c) is applied in areas with the presence of 10%–50% SGS clouds (Texas–New Mexico, Alabama–South Carolina). This case study demonstrates how the SCH assimilation design by neglecting possible presence of SGS clouds can exaggerate the full cloud cover (redundantly combining new explicit and pre-existing SGS cloud) and water vapor. METAR-based cloud building is evident in Figs. 9c and 10c, manifesting often as “disk-like” regions (e.g., in South Dakota). The disk shapes occur where the surface data are sparse under a higher-level cloud deck blocking satellite horizontal refinement; this aspect needs further work in the future. Smaller-scale detail from GOES where high cloud is absent and prior 1-h forecast cloud detail partially modifies these increments into non-disk-shaped refinements in North Dakota–South Dakota–Minnesota in Fig. 9c [refer to section 3b(1)].
e. SCH assimilation within the RAP and HRRR assimilation process
SCHDA is applied within the assimilation cycle flow (Fig. 11) for both the RAP and HRRR models. SCHDA is performed within GSI after the variational-ensemble hybrid assimilation component is complete.
Flow diagram for the hourly updated Rapid Refresh (v5) and HRRR (v4) models (2020) including the stratiform cloud-hydrometeor (SCH) analysis step within GSI. [Updated from Fig. 2 in B16 for the 2015 version of the Rapid Refresh. Also see Dowell et al. (2021, manuscript submitted to Mon. Wea. Rev.).]
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
The analyzed hydrometeor fields are also later reapplied within the Digital Filter Initialization (DFI) step used only for the RAP model (Fig. 11). Peckham et al. (2016) describe the DFI application to the RAP model and its overall beneficial effect on reducing noise by achieving a wind-mass adjustment (also, Benjamin et al. 2004a, Fig. 2). Backward (adiabatic, inviscid) and forward (diabatic, full physics) integrations of 20 min are performed with digital-filter-weighting averages resulting in more balanced mass, wind, and water vapor fields. To allow retention of SCHDA cloud fields in the WRF model forecast after invoking DFI, the 3D qc–qi array is saved in the forecast model before DFI. As a side effect, DFI decreases the fraction of 3D grid points with saturation (from its time-step averaging with a one-sided limit preventing water vapor (qv) values over saturation). The previously saved 3D qc–qi array is then reintroduced after DFI (for RAP but not for HRRR, per Fig. 11) for cloud restoration, including resaturating (setting qυ = qυ-saturation) any cloudy 3D grid volume. After the cloud restoration step, a rebalancing step to reset hydrostatic pressure in WRF has been applied in the RAP (not described in B16). This step was also intended to help ensure retention of clouds. The experiments conducted described in the next section showed that the rebalancing was not needed for short-range forecast clouds and degraded forecast skill through introducing local perturbations and especially for no-SCHDA experiments. Thus, rebalancing is removed in the DFI procedure resulting in 1) better-controlled SCHDA impact experiments described in the next section and 2) a reinterpretation of James and Benjamin (2017, hereafter JB17) results on surface (METAR) and satellite observation impact.
4. Cloud assimilation impact experiments
To demonstrate the forecast impacts of the SCHDA, multiple sets of 11-day retrospective experiments were carried out for summer and winter seasons using the latest version of the RAP system, RAPv5. The test periods assessed are 1–11 February 2019 and 15–25 July 2018, also used for recent aircraft observation impact investigations reported by James et al. (2020). Although the SCHDA runs in both HRRR and RAP, RAP was chosen for these experiments due to its smaller computational expense and adequate resolution at 13 km to capture stratiform clouds. The evaluation of the model output against observations was performed using the Model Analysis Tool Suite (MATS; Turner et al. 2020), where the 95% significance for upper-air verification was calculated using t tests).
Table 3 describes the experiments for quantifying the SCHDA impact, most importantly, with or without inclusion of the SCHDA in both summer and winter periods. The issue on overbuilding clouds (section 3d) is addressed with Exp. SCHDA-45 in which cloud building is reduced by applying it only if background SGS cloud fraction < 45%. Sensitivity to use of a model-rebalancing step after DFI was also considered in other experiments (SCHDA-a and no-SCHDA-a). Separate experiments denying METAR and satellite cloud observations were carried out, respectively, in experiments SatOnly and METARonly.
a. Low-cloud ceiling and visibility forecast impacts
First, we consider skill in low-level cloud ceiling and visibility forecasts, both relatively uncommon (Table 4) but important conditions owing to aviation safety considerations. Ceiling and visibility are diagnosed from model forecasts using combined explicit-cloud and RH criteria described in Benjamin et al. (2020) for aviation forecasts (e.g., Glahn et al. 2017). Low visibility (<3 mi; 1 mi ≈ 1.6 km) diagnosis for HRRR/RAP requires nonzero predicted hydrometeor mixing ratios in the lowest two model levels near the ground (lowest ~30 m AGL) whereas low-level ceiling diagnosis is more relaxed over the lowest several model levels and less dependent on near-surface conditions. Figure 12 shows the hourly frequency of cloud ceiling occurrence below a few key cloud-ceiling thresholds: any cloud, and 3000- (~0.9 km), 1000-, and 500-ft ceilings above ground level (AGL). These curves are based on CONUS METAR ceilometer observations during the 11-day retrospective periods selected for this study. The winter period exhibits overall greater cloudiness than the summer period. The winter period also features stronger synoptic modulation in the cloud statistics, with a period of relatively little cloud cover and almost no diurnal cycle in cloud coverage during 8–10 February 2019. Conversely, the summer period shows little day-to-day variation in cloud coverage, but has a regular diurnal cycle of cloud development and erosion. Frequency of occurrence for low ceiling and low visibility events is summarized in Table 4 for these summer and winter periods. Visibility event frequency for the 1-mile threshold was found to be even rarer than that for 500-ft ceiling, especially in summer.
Frequency of low-ceiling and low-visibility events for the 11-day periods from Jul 2018 (summer) and Feb 2019 (winter) from hourly reports over all 24-h periods from all METAR stations over CONUS.
Hourly occurrence of cloud ceilings for (a) summer and (b) winter periods, based on METAR station observations over the CONUS (~1800 total stations). Ceiling thresholds are shown in feet AGL: 3000, 1000, and 500 ft.
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
Cloud-ceiling forecast performance (Fig. 13) is evaluated against CONUS METAR ceilometer observations using a performance diagram (PD, Roebber 2009) that combines probability of detection (POD) and a success score (1 − false alarm ratio) for the 2 × 2 contingency table information (yes or no for both forecasts and observations). Optimal values (critical success index in black solid curved lines, frequency bias in dashed lines) are toward the upper-right hand corner with maximum detection and minimum false alarm. These diagrams illuminate the trade-offs in POD and bias. Additionally, the relative performance of a given configuration is assessed using Heidke skill score (HSS; Doswell et al. 1990).
Forecast lengths shown for each RAP experiment are 0, 1, 3, 6, 9, and 12 h. Verification metrics for in RAP ceiling forecast skill with forecast lead time for (a),(c) winter and (b),(d) summer retrospective experiments. Verification is against METAR observations over CONUS. (a),(b) Performance diagrams (Roebber 2009) for RAP ceiling forecasts for winter and summer, respectively. Vertical axis in performance diagrams show probability of detection and horizontal axis indicates success rate (1 − false alarm ratio). Results from three experiments (Table 3) are shown: SCHDA (blue), SCHDA-45 (green), and noSCHDA (red). Values are shown for 3000-ft ceilings (circles), 1000-ft ceilings (× marks), and 500-ft ceilings (stars). Verification is against METAR observations over CONUS. (c),(d) Change in HSS vs noSCHDA (solid for SCHDA minus noSCHDA; dotted for SCHDA-45 minus noSCHDA) for winter and summer, respectively. Values are shown for 3000-ft ceilings (red), 1000-ft ceilings (green), and 500-ft ceilings (blue).
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
Figure 13 plots the performance metrics for evaluating the detection of cloud ceilings below 3000, 1000, and 500 ft AGL for the SCHDA, SCHDA-45, and no-SCHDA experiments. POD is plotted against the success rate, (1 − false alarm ratio), in winter (Fig. 13a) and summer (Fig. 13b) for forecasts between 0 and 12 h. The black solid curves denote the overall RAP skill in predicting clouds below a particular ceiling threshold. Enhancements relative to the no-SCHDA experiment (red) are evident in the SCHDA (blue) and the refined SCHDA-45 (green) results for all three ceiling thresholds and both seasons, especially for 1–6-h forecasts and, less so, for 9–12-h forecasts. The increase in frequency bias introduced by the SCHDA, particularly for the lower cloud-ceiling thresholds (1000 and 500 ft), is likely linked to building clouds in the presence of subgrid clouds as discussed in section 3d and demonstrated in Fig. 9d. It is more than offset by the increased POD. Additionally, the high bias (still under 1.8 for most time periods) also augments detection of these relatively rare but hazardous events (Fultz and Ashley 2016). POD for these events is greater with SCHDA out to 12 h in both seasons (Figs. 13a,b). For longer forecasts, the ceiling detection skill with or without SCHDA in these PDs converges toward values constrained by the RAP’s model physics and ceiling diagnostic (Benjamin et al. 2020). The increased skill from SCHDA is more pronounced in winter than in summer and the improvement is strongest for higher ceilings. Finally, the SCHDA modification to limit cloud building where cloud fraction < 45% (experiment SCHDA-45, green in Fig. 13) produces the best overall result in winter. It has little impact during summer, presumably due to the less frequent (Fig. 12), local-scale low clouds occurring in that season.
Differences in HSS relative to the no-SCHDA experiment, plotted for winter (Fig. 13c) and summer (Fig. 13d), are used to focus on the SCHDA impact. HSS is substantially greater for SCHDA than for no-SCHDA during both seasons, with positive differences persisting through, at least, 6-h lead time. Forecast skill as indicated by the HSS is generally smaller for the lower cloud ceiling thresholds, due to the relative rarity of the lower cloud ceiling events (e.g., Fig. 12). Experiment SCHDA-45 increased HSS in winter especially for 1000- and 500-ft ceilings and reduced bias for all three thresholds but had less effect versus SCHDA in summer. Similar ceiling forecast improvement from SCHDA was produced using the post-DFI rebalancing. This illustrated by the summer-only HSS differences between SCHDA-a and noSCHDA-a shown in Fig. 14.
Change in RAP ceiling forecast HSS skill (vs. noSCHDA) during the summer period with forecast lead time for (a) 3000-, (b) 1000-, and (c) 500-ft ceilings. Shown are HSS differences [SCHDA minus noSCHDA (red); SCHDA-a minus noSCHDA-a (blue); see Table 3 for experiment names]. Verification is against METAR observations over CONUS.
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
Additional experiments were conducted to isolate the relative contribution of both METAR (METARonly) and satellite (SatOnly) cloud observations (Table 3). The primary contribution to improved ceiling forecast skill from SCHDA is shown to be from METAR cloud observations (Fig. 15) most clearly for 3000-ft ceiling events. Some improvement from SCHDA was evident at 3- and 6-h durations for 1000-ft ceilings but not so for 1-h forecasts. For the rarer but more hazardous 500-ft events, satellite observations provide skill if METAR cloud observations are not available [cf. Exps. NoSCHDA (i.e., no cloud obs), METARonly, and SatOnly]. If METAR cloud observations are available, then satellite data does not add skill at the METAR stations (used for verification), but it can be inferred that satellite data are adding skill at other locations where METAR data are not available.
Change in RAP ceiling forecast HSS skill (vs noSCHDA) during the winter period with forecast lead time from SCH assimilation and the contributions from individual observation types, for (a) 3000-, (b) 1000-, and (c) 500-ft ceilings. Shown are HSS differences [SCHDA minus noSCHDA (red); SatOnly minus noSCHDA (blue); METARonly minus noSCHDA (green); see Table 3 for experiment names]. Verification is against METAR observations over CONUS.
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
Change to HSS skill in surface visibility forecasts was also examined in these experiments during winter and summer season (Fig. 16), where SCHDA provides a short-range improvement for visibility forecasts out to 6 h in winter and 3 h in summer. Improvement from SCHDA is evident up to 5-mile (8-km) visibility (dependent on lowest-level RH and not on hydrometeor mixing ratios – Benjamin et al. 2020), although not evident for 10-mile (16-km) visibility (not shown). Low-visibility occurrence in the summertime over the lower 48 United States is much rarer than in the winter by a factor of up to 10 (Table 4), making it a yet more challenging forecast problem. POD for these low-visibility events is increased by SCHDA out to 12 h in the winter season (not shown), similar to that for low ceilings. Separate contributions of the METAR and satellite observations on visibility skill were also examined (also Fig. 16) with METAR clouds providing most of the additional visibility forecast skill.
As in Fig. 15, but for change in visibility HSS (vs noSCHDA) for (a),(d) 5-; (b),(e) 3-; and (c),(f) 1-mi visibility for (a)–(c) winter and (d)–(f) summer periods.
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
b. Upper-air forecast impacts
The season-dependent impact of SCHDA is examined for upper-level forecasts of wind, temperature, and relative humidity (Fig. 17 using SCHDA and Fig. 18 using the refined SCHDA-45 technique). During the summer period, low-level temperature (Fig. 17a) and upper-level relative humidity (Fig. 17c) forecasts are improved through application of SCHDA. For the winter period, SCHDA impact is more mixed and often negative, with temperature (significant – Fig. 17b) and wind (not significant, Fig. 17f) forecast degradations evident in the midtroposphere. Upper-level relative humidity forecast improvements (Figs. 17c,d) are seen for both seasons when the SCHDA is used through its cloud clearing. By limiting cloud building in SCHDA to only where cloud fraction < 45% (SCHDA-45, Fig. 18), we see improved humidity forecasts for summer (Fig. 18c) and more neutral impacts on the temperature (Fig. 18a near surface). Negative impact from SCHDA was decreased with SCHDA-45 for wind fields, especially in the winter (Figs. 18b,f). Out of all of these differences in Figs. 17 and 18, the only ones found to be significant are the negative impact from SCHDA on 300–500-hPa temperature and the positive impact on 150–350-hPa RH.
Change in upper-air RAP forecast skill from SCHDA for (a),(c),(e) summer and (b),(d),(f) winter retrospective experiments (SCHDA minus no-SCHDA). Verification is RMS errors vs raob data over CONUS domain for (a),(b) temperature; (c),(d) RH; and (e),(f) wind. Dots indicate statistical significance at the 95% level.
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
As in Fig. 17, but for SCHDA-45 minus no-SCHDA.
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
We add now an incidental paragraph showing the effect on upper-air forecasts only of the post-DFI rebalancing (SCHDA-a vs noSCHDA-a) versus the control experiments (SCHDA, noSCHDA) without rebalancing in Fig. 19. Our experiments confirmed that in both summer and winter seasons, the experiments with rebalancing gave consistently poorer results for wind forecasts (thin black line, Fig. 19a), statistically significant for 250–350 hPa in summer, 250–750 hPa in winter). In particular, the post-DFI cloud restoration/rebalancing with poorer cloud fields (i.e., without SCH assimilation, no SCHDA-a), gave particularly poor results for all variables (wind, temperature, RH), exaggerating the initially apparent positive effect of SCHDA in both summer and winter (see red line). The similar degradation for upper-level temperature and upper-level RH (see Figs. 19c,d for winter only) shows how post-DFI rebalancing greatly exaggerated the preliminary SCHDA beneficial effect. This result indicates that direct application of DFI as described by Peckham et al. (2016) without subsequent rebalancing gives the best balance and avoids introduction of local perturbations from uncertainty in qc–qi fields in the post-DFI rebalancing step available in WRF. The post-DFI water-vapor field without cloud restoration as used in Exp. SCHDA is very effective for producing skillful forecasts as short as 1-h duration (e.g., Figs. 13, 15).
Changes in vertical profiles of 6-h forecast RMSE vs raob data over the CONUS domain for (a) wind during the summer retrospective experiment, (b) wind during the winter retrospective experiment, and (c) temperature and (d) RH during the winter retrospective experiment. Shown are differences in errors compared to experiments SCHDA and SCHDA-a, where positive values indicate larger errors than SCHDA when SCHDA is not run. [Impact of SCHDA in the original experiments, noSCHDA-a minus SCHDA (red); impact of SCHDA in the revised experiments, no SCHDA minus SCHDA (blue); impact of rebalancing, SCHDA-a minus SCHDA (black); see Table 3. 95% significance shown].
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
We now conclude that this effect also exaggerated the earlier result from using satellite data (including clouds) and surface data (also including METAR ceilometer clouds) found by JB17. We also conclude that the RAP configuration without the post-DFI rebalancing (SCHDA as shown in Table 3) is the best configuration for RAP performance and for these experiments. The operational RAP through RAPv5 used the configuration with restoration/rebalancing similar to that used for SCHDA-a (Expt. 1a).3
c. Surface weather forecast impacts
The impact of SCHDA on 2-m temperature forecasts was found to be slightly positive in in the daytime for both seasons (Fig. 20). At night, SCHDA had no impact for 2-m temperature or dewpoint in summer but negative for both variables at night in winter. In summer, SCHDA reduces daytime RMS errors by 0.02–0.05 K for forecasts of both variables valid in daytime, which would be important for the preconvective environment. All of these changes were slight, not statistically significant at the 95% level. Impact on precipitation forecast skill (not shown) from SCHDA was very small.
2-m (a),(b) temperature and (c),(d) dewpoint (RMS error) RAP forecast impact from SCH assimilation for (a),(c) summer and (b),(d) winter using SCHDA minus no-SCHDA retrospective experiments and for eastern United States vs METAR observations.
Citation: Monthly Weather Review 149, 8; 10.1175/MWR-D-20-0319.1
Overall in this section, these tests of the SCH assimilation during summer and winter seasons confirm a positive impact for short-range (3–9 h) ceiling and visibility forecasts in both seasons. Upper-air and surface verification showed more mixed impacts, with a modest improvement in the summer test period (Figs. 18c,e and 18a near surface; also Figs. 20a,c) but a slight degradation in winter (Figs. 18b,f; Figs. 20b,d). An improvement in RH forecasts in the upper troposphere occurs from SCHDA in both seasons, and in lower-tropospheric temperature in the summer experiment. A new experiment with more constrained cloud building using background subgrid-scale cloud fraction gave improved results overall, especially in winter, including a more neutral bias.
5. Conclusions
Short-range forecasts of stratiform cloud are extremely important for a wide range of applications, including aviation, ground transportation, and solar energy. The SCH analysis technique described herein represents an effective way to initialize stratiform cloud and hydrometeors in a rapidly updating mesoscale or storm-scale model. A realistic treatment of clouds during the first few hours of an NWP forecast enables a slightly improved prediction of daytime 2-m temperature and moisture. Although not demonstrated here, this should lead to improved daytime boundary layer evolution, with associated benefits to forecasts of convective evolution. The NOAA operational RAP and HRRR NWP systems make use of the SCHDA technique to improve accuracy of real-time forecasts. The SCHDA is also being tested in initial versions of the FV3-based 3-km Rapid Refresh Forecast System planned to replace the HRRR by 2024.
This article serves as a documentation of the full SCH analysis technique. Information on 3D cloud location from remote sensing instruments (both geostationary satellites and ground-based) is used in combination with prior information from a previous model forecast in order to clear and build cloudy volumes within the model. Temperature and moisture are also locally adjusted in a manner consistent with the presence or absence of clouds through conserving virtual potential temperature. The technique checks for consistency among the various observational sources of information (Table 2) and avoids introducing moist instability into the model initial conditions.
The SCH assimilation technique has a beneficial impact on low-cloud-ceiling and low-visibility forecasts in both summer and winter although also introducing a high bias for these rare and hazardous events. According to separate METAR and satellite cloud denial experiments in this study, the generally positive result appears to be primarily from assimilation of METAR ceilometer and visibility data but satellite-cloud-only assimilation improves some low-ceiling forecasts events. Smaller but still statistically significant positive impacts from SCHDA are also shown for upper-air RH forecasts in summer and winter, with small subsignificant positive impacts on low-level temperature in summer and winter but a negative impact on midlevel temperature in both seasons. The overall SCHDA impact highlights the importance of these observational datasets, satellite cloud retrievals and METAR ceilometer and visibility observations, for application within NWP systems to enable more skillful short-range forecasts of stratiform clouds and hydrometeors. An SCHDA refinement to build clouds only when subgrid-scale cloud fraction is sufficiently small (SCHDA-45) was tested and improved results, especially in winter.
A previous observation impact study by James and Benjamin (2017) found a surprisingly substantial impact aloft for wind, temperature and RH forecasts from satellite and surface observations (their Fig. 9 for 800–400 and 400–100 hPa), apparently related to the SCH assimilation. Based on further investigations described in this article, the impact indicated in JB17 for satellite (including cloud top) and surface (METAR; including ceiling) was found to be inflated and a consequence of a suboptimal configuration of the RAP initialization involving rebalancing and restoration of hydrometeors after digital filter initialization. In this paper, we examine the impact of the initialization procedure and provide a more appropriate quantification of the SCHDA impact in a more optimal configuration of RAP initialization, much smaller than implied in JB17.
The results of this study are dependent on assumptions for SCHDA parameters and also on the particular ceiling and visibility diagnostic methods used. Future improved estimation of these values and new diagnostics (see section 2b) may give different results. Assumed SCHDA parameter values (cloud depth, spatial representativeness of METAR data, RH for subsaturation, qc–qi values for building, clearing cloud fraction threshold) can be better estimated, likely through ensemble DA methods. Recent developments within the MYNN turbulence scheme (e.g., Olson et al. 2019a,b; Angevine et al. 2018) to improve explicit representation of subgrid-scale cloudiness allowed us to use background subgrid cloud-fraction information in SCHDA. This was explored with the SCHDA-45 experiment (Table 3) in this study showing some improvement (Figs. 13, 18) but further refinement based on season or cloud scale is needed. The assumption so far in SCHDA of saturation in a 3D-grid point for existence of clouds needs to be relaxed in part through assimilation of nonbinary cloud fraction using cloud fraction as a prognostic variable using ensemble DA. Future revision of ceiling or visibility diagnostics using subgrid cloud-fraction may also change these results.
Several avenues of future research hold potential for improvements in SCH initialization in NWP systems. Cloud retrievals from polar-orbiting satellites can also be used in SCHDA to benefit forecasts especially at high-latitude regions. Additional observation sources such as air or ground traffic cameras (e.g., FAA-funded cameras4 or all-sky cameras (Long and DeLuisi 1998)) could also be applied in the context of the SCH analysis to expand the number of available observations. Binary cloud–no-cloud observations such as those described herein are being used as “proxy” water vapor observations and tested with hybrid ensemble–variational DA techniques. The SCH assimilation procedure described in this paper represents an important baseline for measuring the performance of more advanced assimilation procedures which will be investigated in the future and remains a beneficial method for initializing low clouds for short-range forecasting.
Acknowledgments
The RAP and HRRR weather prediction models and this stratiform cloud/hydrometeor assimilation technique were developed under funding from NOAA Research [including its Atmospheric Science for Renewable Energy (ASRE) program], the Federal Aviation Administration (FAA), and the Department of Energy. This research is in response, in part, to requirements from and funding by the FAA. The views expressed are those of the authors and do not necessarily represent the official policy or position of the FAA. We are grateful to Rabindra Palikonda (NASA Langley) for coordinating with NOAA/OAR and NWS in providing GOES-based cloud-retrieval data. Thanks to Greg Thompson and two anonymous reviewers for excellent formal reviews. We also thank Hongli Wang from NOAA/GSL for a very helpful review of the manuscript and the NOAA HPC and NOAA/GSL/ITS staff for providing computing resources for this work. David Dowell (GSL) developed the cloud-water/ice diagnostic method from reflectivity, and Craig Hartsough (GSL) provided Fig. 7. W. Smith and P. Minnis are supported by the NASA Modeling and Analysis Program (MAP) and the NASA R&A Weather Focus Area.
Data availability statement
The observations and other data used in this research are housed within the NOAA/Global Systems Laboratory (GSL) database. The SatCORPS GOES retrieval data used in this study are available at https://satcorps.larc.nasa.gov.
REFERENCES
Albers, S. C., J. A. McGinley, D. A. Birkenheuer, and J. R. Smart, 1996: The Local Analysis and Prediction System (LAPS): Analysis of clouds, precipitation and temperature. Wea. Forecasting, 11, 273–287, https://doi.org/10.1175/1520-0434(1996)011<0273:TLAAPS>2.0.CO;2.
Angevine, W. M., J. B. Olson, J. Kenyon, W. I. Gustafson Jr., S. Endo, K. Suselj, and D. D. Turner, 2018: Shallow cumulus in WRF parameterizations evaluated against LASSO large-eddy simulation. Mon. Wea. Rev., 146, 4303–4322, https://doi.org/10.1175/MWR-D-18-0115.1.
Asadi, N., K. A. Scott, and D. A. Clausi, 2019: Data fusion and data assimilation of ice thickness observations using a regularisation framework. Tellus, 71A, 1564487, https://doi.org/10.1080/16000870.2018.1564487.
Atkinson, D., 2018: Expanded cloud base and backscatter detection using the ASOS ceilometer. Sixth Symp. on the Weather, Water, and Climate Enterprise, Austin, TX, Amer. Meteor. Soc., 288, https://ams.confex.com/ams/98Annual/webprogram/Paper327129.html.
Bauer, P., A. J. Geer, P. Lopez, and D. Salmond, 2010: Direct 4D-Var assimilation of all-sky radiances. Part I: implementation. Quart. J. Roy. Meteor. Soc., 136, 1868–1885, https://doi.org/10.1002/qj.659.
Bayler, G. M., R. M. Aune, and W. H. Raymond, 2000: NWP cloud initialization using GOES sounder data and improved modeling of nonprecipitating clouds. Mon. Wea. Rev., 128, 3911–3920, https://doi.org/10.1175/1520-0493(2001)129<3911:NCIUGS>2.0.CO;2.
Benjamin, S. G., and T. N. Carlson, 1986: Some effects of surface heating and topography on the regional severe storm environment. Part I: Three-dimensional simulations. Mon. Wea. Rev., 114, 307–329, https://doi.org/10.1175/1520-0493(1986)114<0307:SEOSHA>2.0.CO;2.
Benjamin, S. G., and Coauthors, 2004a: An hourly assimilation/forecast cycle: The RUC. Mon. Wea. Rev., 132, 495–518, https://doi.org/10.1175/1520-0493(2004)132<0495:AHACTR>2.0.CO;2.
Benjamin, S. G., J. M. Brown, S. S. Weygandt, T. L. Smith, B. Schwartz, and W. R. Moninger, 2004b: Assimilation of surface cloud, visibility, and current weather observations in the RUC. 20th Conf. on Weather Analysis and Forecasting/16th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., P1.47, https://ams.confex.com/ams/84Annual/techprogram/paper_71242.htm.
Benjamin, S. G., and Coauthors, 2016: A North American hourly assimilation and model forecast cycle: The Rapid Refresh. Mon. Wea. Rev., 144, 1669–1694, https://doi.org/10.1175/MWR-D-15-0242.1.
Benjamin, S. G., E. P. James, J. M. Brown, E. J. Szoke, J. S. Kenyon, and R. Ahmadov, 2020: Diagnostic fields developed for hourly updated NOAA weather models. NOAA Tech. Memo. OAR GSL-66, https://doi.org/10.25923/98fy-xx71.
Boudala, F. S., G. A. Isaac, R. W. Crawford, and J. Reid, 2012: Parameterization of runway visual range as a function of visibility: Implications for numerical weather prediction models. J. Atmos. Oceanic Technol., 29, 177–191, https://doi.org/10.1175/JTECH-D-11-00021.1.
Carlson, T. N., S. G. Benjamin, G. S. Forbes, and Y.-F. Li, 1983: Elevated mixed layers in the regional severe storm environment: Conceptual model and case studies. Mon. Wea. Rev., 111, 1453–1473, https://doi.org/10.1175/1520-0493(1983)111<1453:EMLITR>2.0.CO;2..
Colle, B. A., K. J. Westrick, and C. F. Mass, 1999: Evaluation of MM5 and Eta-10 precipitation forecasts over the Pacific Northwest during the cool season. Wea. Forecasting, 14, 137–154, https://doi.org/10.1175/1520-0434(1999)014<0137:EOMAEP>2.0.CO;2.
Corfidi, S. F., S. J. Corfidi, and D. M. Schultz, 2008: Elevated convection and castellanus: Ambiguities, significance, and questions. Wea. Forecasting, 23, 1280–1303, https://doi.org/10.1175/2008WAF2222118.1.
Crum, T. D., R. L. Alberty, and D. W. Burgess, 1993: Recording, archiving, and using WSR-88D data. Bull. Amer. Meteor. Soc., 74, 645–653, https://doi.org/10.1175/1520-0477(1993)074<0645:RAAUWD>2.0.CO;2.
de Haan, S., and S. H. van der Veen, 2014: Cloud initialization in the Rapid Update Cycle of HIRLAM. Wea. Forecasting, 29, 1120–1133, https://doi.org/10.1175/WAF-D-13-00071.1.
Donner, L. J., and P. J. Rasch, 1989: Cumulus initialization in a global model for numerical weather prediction. Mon. Wea. Rev., 117, 2654–2671, https://doi.org/10.1175/1520-0493(1989)117<2654:CIIAGM>2.0.CO;2.
Doswell, C. A., R. Davies-Jones, and D. L. Keller, 1990: On summary measures of skill in rare event forecasting based on contingency tables. Wea. Forecasting, 5, 576–585, https://doi.org/10.1175/1520-0434(1990)005<0576:OSMOSI>2.0.CO;2.
Foster, M. J., and A. Heidinger, 2014: Entering the era of +30-year satellite cloud climatologies: A North American case study. J. Climate, 27, 6687–6697, https://doi.org/10.1175/JCLI-D-14-00068.1.
Fultz, A. J., and W. S. Ashley, 2016: Fatal weather-related general aviation accidents in the United States. Phys. Geogr., 37, 291–312, https://doi.org/10.1080/02723646.2016.1211854.
Glahn, B., A. D. Schnapp, J. E. Ghirardelli, and J.-S. Im, 2017: A LAMP-HRRR MELD for improved aviation guidance. Wea. Forecasting, 32, 391–405, https://doi.org/10.1175/WAF-D-16-0127.1.
Häggmark, L., K.-I. Ivarsson, S. Gollvik, and P.-O. Olofsson, 2000: Mesan, an operational mesoscale analysis system, Tellus, 52A, 2–20, https://doi.org/10.3402/tellusa.v52i1.12250.
Hu, M., S. G. Benjamin, T. T. Ladwig, D. C. Dowell, S. S. Weygandt, C. R. Alexander, and J. S. Whitaker, 2017: GSI three-dimensional ensemble-variational hybrid data assimilation using a global ensemble for the regional Rapid Refresh model. Mon. Wea. Rev., 145, 4205–4225, https://doi.org/10.1175/MWR-D-16-0418.1.
James, E. P., and S. G. Benjamin, 2017: Observation system experiments with the hourly updating Rapid Refresh model using GSI hybrid ensemble-variational data assimilation. Mon. Wea. Rev., 145, 2897–2918, https://doi.org/10.1175/MWR-D-16-0398.1.
James, E. P., S. G. Benjamin, and B. D. Jamison, 2020: Commercial aircraft-based observations for NWP: Global coverage, data impacts, and COVID-19. J. Appl. Meteor. Climatol., 59, 1809–1825, https://doi.org/10.1175/JAMC-D-20-0010.1.
Kleist, D. T., D. F. Parrish, J. C. Derber, R. Treadon, W.-S. Wu, and S. Lord, 2009: Introduction of the GSI into the NCEP global data assimilation system. Wea. Forecasting, 24, 1691–1705, https://doi.org/10.1175/2009WAF2222201.1.
Kollias, P., C. W. Fairall, P. Zuidema, J. Tomlinson, and G. A. Wick, 2004: Observation of marine stratocumulus in SE Pacific during the PACS 2003 cruise. Geophys. Res. Lett., 31, L22110, https://doi.org/10.1029/2004GL020751.
Li, J., W. P. Menzel, and A. J. Schreiner, 2001: Variational retrieval of cloud parameters from GOES sounder longwave cloudy radiance measurements. J. Appl. Meteor., 40, 312–330, https://doi.org/10.1175/1520-0450(2001)040<0312:VROCPF>2.0.CO;2.
Long, C. N., and J. J. DeLuisi, 1998: Development of an automated hemispheric sky imager for cloud fraction retrievals. Preprints, Tenth Symp. on Meteorological Observations and Instrumentation, Phoenix, AZ, Amer. Meteor. Soc., 171–174.
Macpherson, B., B. J. Wright, W. H. Hand, and A. J. Maycock, 1996: The impact of MOPS moisture data in the U.K. Meteorological Office mesoscale data assimilation scheme. Mon. Wea. Rev., 124, 1746–1766, https://doi.org/10.1175/1520-0493(1996)124<1746:TIOMMD>2.0.CO;2.
Marshall, J. S., and W. M. Palmer, 1948: The distribution of raindrops with size. J. Meteor., 5, 165–166, https://doi.org/10.1175/1520-0469(1948)005<0165:TDORWS>2.0.CO;2.
McNally, A. P., 2009: The direct assimilation of cloud-affected satellite infrared radiances in the ECMWF 4D-Var. Quart. J. Roy. Meteor. Soc., 135, 1214–1229, https://doi.org/10.1002/qj.426.
Migliorini, S., and B. Candy, 2019: All-sky satellite data assimilation of microwave temperature sounding channels at the Met Office. Quart. J. Roy. Meteor. Soc., 145, 867–883, https://doi.org/10.1002/qj.3470.
Minnis, P., and Coauthors, 2008: Near-real time cloud retrievals from operational and research meteorological satellites. Proc. SPIE, 7107, 710703, https://doi.org/10.1117/12.800344.
Minnis, P., and Coauthors, 2020: CERES MODIS cloud product retrievals for Edition 4—Part I: Algorithm changes. IEEE Trans. Geosci. Remote Sens., 59, 2744–2780, https://doi.org/10.1109/TGRS.2020.3008866.
Münkel, C., N. Eresmaa, J. Räsänen, and A. Karppinen, 2007: Retrieval of mixing height and dust concentration with lidar ceilometer. Bound.-Layer Meteor., 124, 117–128, https://doi.org/10.1007/s10546-006-9103-3.
O’Connor, E. J., A. J. Illingworth, and R. J. Hogan, 2004: A technique for auto-calibration of cloud lidar. J. Atmos. Oceanic Technol., 21, 777–786, https://doi.org/10.1175/1520-0426(2004)021<0777:ATFAOC>2.0.CO;2.
Okamoto, K., A. P. McNally, and W. Bell, 2014: Progress towards the assimilation of all-sky infrared radiances: An evaluation of cloud effects. Quart. J. Roy. Meteor. Soc., 140, 1603–1614, https://doi.org/10.1002/qj.2242.
Olson, J. B., and Coauthors, 2019a: Improving wind energy forecasting through numerical weather prediction model development. Bull. Amer. Meteor. Soc., 100, 2201–2220, https://doi.org/10.1175/BAMS-D-18-0040.1.
Olson, J. B., J. S. Kenyon, W. M. Angevine, J. M. Brown, M. Pagowski, and K. Sušelj, 2019b: A description of the MYNN-EDMF scheme and the coupling to other components in WRF-ARW. NOAA Tech. Memo. OAR GSD-61, https://doi.org/10.25923/n9wm-be49.
Pal, S. R., W. Steinbrecht, and A. I. Carswell, 1992: Automated method for lidar determination of cloud-base height and vertical extent. Appl. Opt., 31, 1488–1494, https://doi.org/10.1364/AO.31.001488.
Peckham, S. E., T. G. Smirnova, S. G. Benjamin, J. M. Brown, and J. S. Kenyon, 2016: Implementation of a digital filter initialization in the WRF model and application in the Rapid Refresh. Mon. Wea. Rev., 144, 99–106, https://doi.org/10.1175/MWR-D-15-0219.1.
Renshaw, R., and P. N. Francis, 2011: Variational assimilation of cloud fraction in the operational Met Office Unified Model. Quart. J. Roy. Meteor. Soc., 137, 1963–1974, https://doi.org/10.1002/qj.980.
Roebber, P., 2009: Visualizing multiple measures of forecast quality. Wea. Forecasting, 24, 601–608, https://doi.org/10.1175/2008WAF2222159.1.
Skamarock, W. C., and Coauthors, 2008: A description of the Advanced Research WRF version 3. NCAR Tech. Note NCAR/TN-475+STR, 113 pp., https://doi.org/10.5065/D68S4MVH.
Stoelinga, M., and T. T. Warner, 1999: Nonhydrostatic, mesobeta-scale model simulations of cloud ceiling and visibility for an East Coast winter precipitation event. J. Appl. Meteor., 38, 385–404, https://doi.org/10.1175/1520-0450(1999)038<0385:NMSMSO>2.0.CO;2.
Thompson, G., and T. Eidhammer, 2014: A study of aerosol impacts on clouds and precipitation development in a large winter cyclone. J. Atmos. Sci., 71, 3636–3658, https://doi.org/10.1175/JAS-D-13-0305.1.
Thompson, G., R. Bullock, and T. F. Lee, 1997: Using satellite data to reduce spatial extent of diagnosed icing. Wea. Forecasting, 12, 185–190, https://doi.org/10.1175/1520-0434(1997)012<0185:USDTRS>2.0.CO;2.
Thompson, G., P. R. Field, R. M. Rasmussen, and W. D. Hall, 2008: Explicit forecasts of winter precipitation using an improved bulk microphysics scheme. Part II: Implementation of a new snow parameterization. Mon. Wea. Rev., 136, 5095–5115, https://doi.org/10.1175/2008MWR2387.1.
Trepte, Q. Z., and Coauthors, 2019: Global cloud detection for CERES Edition 4 using Terra and Aqua MODIS data. IEEE Trans. Geosci. Remote Sens., 57, 9410–9449, https://doi.org/10.1109/TGRS.2019.2926620.
Turner, D. D., and Coauthors, 2020: A verification approach used in developing the Rapid Refresh and other numerical weather prediction models. J. Oper. Meteor., 8, 39–53, https://doi.org/10.15191/nwajom.2020.0803.
Weygandt, S. S., and S. G. Benjamin, 2007: Radar reflectivity-based initialization of precipitation systems using a diabatic digital filter within the Rapid Update Cycle. Preprints, 22nd Conf. on Weather Analysis and Forecasting/18th Conf. on Numerical Weather Prediction, Park City, UT, Amer. Meteor. Soc., 1B.7, https://ams.confex.com/ams/22WAF18NWP/techprogram/paper_124540.htm.
Weygandt, S. S., S. G. Benjamin, T. G. Smirnova, and J. M. Brown, 2008a: Assimilation of radar reflectivity data using a diabatic digital filter within the Rapid Update Cycle. Preprints, 12th Conf. on IOAS-AOLS, New Orleans, LA, Amer. Meteor. Soc., 8.4, https://ams.confex.com/ams/88Annual/techprogram/paper_134081.htm.
Weygandt, S. S., S. G. Benjamin, M. Hu, T. G. Smirnova, and J. M. Brown, 2008b: Use of lightning data to enhance radar assimilation within the RUC and Rapid Refresh models. Third Conf. on Meteorological Applications of Lightning Data, New Orleans, LA, Amer. Meteor. Soc., 8.4, http://ams.confex.com/ams/88Annual/techprogram/paper_134112.htm.
Wood, R., 2012: Stratocumulus clouds. Mon. Wea. Rev., 140, 2373–2423, https://doi.org/10.1175/MWR-D-11-00121.1.
Wyngaard, J. C., 1985: Structure of the planetary boundary layer and implications for its modeling. J. Climate Appl. Meteor., 24, 1131–1142, https://doi.org/10.1175/1520-0450(1985)024<1131:SOTPBL>2.0.CO;2.
Zhu, Y., G. Gayno, R. J. Purser, X. Su, and R. Yang, 2019: Expansion of the all-sky radiance assimilation to ATMS at NCEP. Mon. Wea. Rev., 147, 2603–2620, https://doi.org/10.1175/MWR-D-18-0228.1.
See common Terminal Aerodrome Forecast (TAF) and METAR format information at https://www.aviationweather.gov/static/help/taf-decode.php.
The Vaisala CL31 ceilometer (Münkel et al. 2007) now used in the United States has a range of 25 000 ft but currently, most METAR reports only include data up to 12 000 ft, per Atkinson (2018).
Discovery of this issue was too late in the NWS implementation process to include this change in RAPv5.