1. Introduction
Assimilation of satellite data into global numerical weather prediction (NWP) models has led to substantial forecast improvements during the past two decades (e.g., Derber and Wu 1998; McNally et al. 2000, 2006; Auligné et al. 2011; Zhu et al. 2016). As new satellites and sensors are launched, the additional data has continued this trend in increasing forecast skill to this day. While satellite data has proven to be a vital tool in global NWP, its impact to high-resolution, regional NWP systems is still being assessed. Data assimilation into regional models such as the High-Resolution Rapid Refresh (HRRR) contains many challenges not present in global systems. The HRRR runs over a North American domain at a 3 km horizontal resolution with hourly data assimilation cycling (Benjamin et al. 2016; Alexander et al. 2018). At these temporal and spatial resolutions, many of the assumptions applied in global satellite data assimilation, such as error correlation, data thinning, quality control and handling of outliers are not necessarily applicable on the convection allowing domain. In particular, the poor spatial data coverage of polar orbiting sensors, which have the greatest impact in global models, significantly limits their potential impact in higher resolution regional models. Data from geostationary satellites such as the Advanced Baseline Imager (ABI) on board the GOES-R series are much more suitable to the requirements of these systems. The ABI samples far fewer channels than polar orbiting hyperspectral sounders (16 versus ~1000), but the channels it does sample have important sensitivities to atmospheric temperature and moisture properties and are available at a 2 km horizontal resolution every 10–15 min with a data latency on the order of a few minutes (Schmit et al. 2005). The low data latency is very important for the Warn-on-Forecast System (WoFS), which assimilates data at 15 min intervals (or less) over a regional domain in a real-time fashion to generate short-term (0–6 h) forecasts of high-impact weather events (Stensrud et al. 2009, 2013; Gallo et al. 2017; Choate et al. 2018).
In recent years, many studies have been performed to assess the suitability of geostationary satellite products into convection allowing models, which have shown great promise (e.g., Szyndel et al. 2005; Vukicevic et al. 2006; Stengel et al. 2009; Polkinghorne et al. 2010; Polkinghorne and Vukicevic 2011; Otkin 2012a,b; Qin et al. 2013; Zou et al. 2013, 2015; Jones et al. 2013, 2014, 2015, 2016; Zhang et al. 2016; Minamide and Zhang 2019; Honda et al. 2018a,b; Zhang et al. 2018; Okamoto et al. 2019; F. Zhang et al. 2019; Y. Zhang et al. 2019). One key advantage of satellite data is its availability in regions where other surface and radar observations are not reliably present. Thus, many of these studies have focused on assimilating geostationary satellite data to improve hurricane track and intensity forecasts (e.g., Zou et al. 2015; Zhang et al. 2016; Minamide and Zhang 2019; Honda et al. 2018a,b; F. Zhang et al. 2019). Others have focused on severe weather prediction over land when other data sources are not available (e.g., Zhang et al. 2018). Finally, several studies assimilated satellite data in concert with other high-resolution datasets such as radar reflectivity and radial velocity to complement the advantages of each to increase skill in high-impact weather prediction (Jones et al. 2013, 2015, 2016, 2018; Y. Zhang et al. 2019).
Assimilation of surface-based radar reflectivity and radial velocity observations forms the basis for WoFS-like systems (Aksoy et al. 2009, 2010; Dowell et al. 2011; Yussouf et al. 2013, 2015; Wheatley et al. 2015; Johnson et al. 2015; Wang and Wang 2017). However, satellite data samples nonprecipitating clouds and environmental conditions not readily sensed from radars. Assimilating these data provides this information to the model analysis, often improving forecasts (Jones et al. 2015, 2016). Satellite data are also useful in providing information on convection in the absence of radars, but a truly successful WoFS system does require reasonable radar data coverage to reliably generate consistently skillful forecasts.
Satellite data comes in many forms and can be assimilated as radiances (brightness temperatures) or retrievals. Each method has its advantages and disadvantages, but both convey important environmental and cloud properties to the data assimilation system. Jones et al. (2015, 2016) assimilated cloud water path (CWP) retrievals into the WoFS, and showed improvement in the forecasting of cloud properties, convective initiation, and the near-storm environment compared to experiments that only assimilated radar data. Similar results were obtained by Zhang et al. (2018) through assimilating all-sky water vapor channel radiances. Jones et al. (2018) experimented with assimilating GOES-13 6.95 μm clear-sky water vapor channel radiances in combination with radar and CWP and showed that assimilating radiances did improve the model analysis when compared against observations. This translated to improvements in the forecasting of rotating severe storms, but the correction of inherent model biases caused a degradation in one case.
With the operational availability of GOES-16 data and the increasing maturity of the WoFS, a comprehensive analysis of the relative impact of CWP retrievals, clear-sky, and all-sky radiances is necessary. However, the ideal set of observations to assimilate remains an open question and this work hopes to provide some answers using several high-impact severe weather events run in real time during spring and summer 2019. In particular, comparing the results from assimilating retrieved cloud properties versus all-sky radiances is necessary to determine the advantages and disadvantages of both observation types. Migliorini (2012) found that in the end, both contain a similar information content, but their observation characteristics differ significantly, which can have large impacts during the assimilation processes. For example, assimilating retrievals or radiances associated with upper-level cirrus clouds both add positive increments to frozen hydrometeor values. However, the magnitude of these increments can vary substantially along with the impact to specific hydrometeor variables. Testing showed that assimilating CWP observations had the largest impact on snow concentrations while assimilating radiances had the largest impact on ice concentrations in the same atmospheric layer (not shown).
An experiment that assimilates only radar observations will be used as a baseline from which the value of satellite data will be assessed. Qualitative and object-based verification of severe weather and the near-storm environment will be used to assess which combination of observations generates the most skillful forecasts (Skinner et al. 2018; Jones et al. 2018). We will focus our study through the entire WoFS analysis and forecasting cycle (1900–0600 UTC, daily) so that the impacts throughout the evolution of convection from initiation to large upscale growth can be assessed.
Following the Introduction, section 2 discusses the WoFS configuration and assimilated observations. Brief descriptions of the severe weather events being forecasts are provided in section 3. Section 4 outlines bias and error for each observation type. Section 5 describes qualitative and quantitative comparisons of each assimilation experiment, with conclusions following in section 6.
2. Warn-on-Forecast System (WoFS)
a. Overview
The WoFS is an ensemble data assimilation and forecasting system designed to generate short-term (0–6 h) forecasts of severe thunderstorm, high winds, supercell rotation, and flash flooding. The WoFS uses an ensemble Kalman filter (EnKF) approach to assimilate conventional, radar, and satellite data on a 3 km horizontal resolution, 51 vertical level in a regional domain (Wheatley et al. 2015; Jones et al. 2016; Skinner et al. 2018; Yussouf and Knopfmeier 2019). Currently (2019) the WoFS uses a modified version the of Advanced Weather Research and Forecasting Model (WRF-ARW), version 3.8.1 (Skamarock et al. 2008), coupled with a customized version of the Community Gridpoint Statistical Interpolation (GSI) system that contains the forward operators and data assimilation code (e.g., Kleist et al. 2009; Hu et al. 2016). To generate satellite radiances, GSI uses the Community Radiative Transfer Model (CRTM), which is a tool that translates model state variables into simulated radiances for comparison with observations (Weng 2007; Han et al. 2007). The GSI-EnKF system has been extended to include radar reflectivity and radial velocity (Johnson et al. 2015; Wang and Wang 2017), CWP (Jones et al. 2013), dewpoint, and GOES-16 ABI forward operators using CRTM, version 2.3, as part of ongoing research.
All observations are assimilated using an ensemble Kalman filter (EnKF) approach so that the flow dependent covariances generated by the ensemble after each assimilation cycle can be used in updating the model state (Whitaker et al. 2008). The WoFS cycles at 15 min intervals beginning at 1700 UTC until 0300 UTC assimilating all available conventional, radar, and satellite observations during this period into a 36 member ensemble. Initial and boundary conditions are provided by an experimental 36 member HRRR ensemble (HRRRE; Benjamin et al. 2016) using 1 h forecasts from the 1600 UTC analysis and forecasts generated from the first 9 members of the 1200 UTC cycle, respectively. For 2019, the WoFS uses a regional domain of 300 × 300 grid points (~900 km × 900 km) nested within the HRRRE, which is centered within the area where high-impact weather is expected to occur on a particular day. All ensemble members use the two-moment NSSL variable density (NVD) cloud microphysics scheme, with adjustments to reduce upper-level cloud biases applied (Ziegler 1985; Mansell et al. 2010; Jones et al. 2018). This differs from the HRRRE, which uses the Thompson cloud microphysics scheme for all ensemble members (Thompson et al. 2004, 2008, 2016). During each cycle, temperature, humidity, 3D wind, pressure, diabatic heating, and hydrometeor variables are updated.
Ensemble spread is maintained by applying different sets of model boundary layer physics and radiation schemes to each member (Stensrud et al. 2000). See Table 2 and Table 1 in Wheatley et al. (2015) and Skinner et al. (2018), respectively, for details. Prior adaptive inflation using the Anderson (2009) technique, which has been extended to this system, is applied prior to each assimilation cycle (Hu et al. 2019). An outlier threshold of 3.25 standard deviations from the mean is applied to all observations, similar to (Wheatley et al. 2015; Jones et al. 2018). Horizontal and vertical localizations applied using the Gaspari and Cohn (1999) method are varied as a function of observation type. Conventional observations having the longest (460 km) and high density radar data having the smallest (18 km) localization length are similar to those used by Jones et al. (2018). Clear- and all-sky radiance localization and observation errors are derived from sensitivity testing as well as results from Jones et al. (2015), Honda et al. (2018b), and Y. Zhang et al. (2019). See Table 1 for a complete listing.
Observation errors and localization radii for all observation types assimilated into this version of WoFS. Horizontal localization radii vary for conventional observations being shortest for Oklahoma Mesonet observations, and longest for sparser resolution instruments such as ASOS and ACARS. Vertical localization radii are given in units of scale height. For cloudy regions, BT73 errors are double to account for larger uncertainties in these measurements.
b. Observations
1) Conventional
The WoFS assimilates conventional observations (temperature, dewpoint, winds, and pressure) from surface instruments, aircraft, and radiosondes. Most conventional observations are contained in hourly prepbufr files also used by the HRRRE system and assimilated into the WoFS when available using a 15 min time lag. For domains that include Oklahoma (OK), OK Mesonet data are also assimilated at each cycle to compliment other conventional observations in the prepbufr file (McPherson et al. 2007). Observation errors for these and all other observation types are provided in Table 1.
2) Radar reflectivity and radial velocity
Reflectivity observations are derived from the 1-km Multi-Radar Multi-Sensor (MRMS) product created from the WSR-88D Doppler radar network that are objectively analyzed to a 5-km resolution (Smith et al. 2016). Vertical resolution is 0.5 km from the surface to 3 km above sea level and 1 km thereafter until 10 km above sea level. Reflectivity values between 0 and 15 dBZ are not assimilated to provide a buffer between precipitation and nonprecipitation regions, which are defined as 0 dBZ. Any negative reflectivity values are set to zero during the MRMS preprocessing phase. For clear-air reflectivity, only a single value per grid point is assimilated and the data are further thinned to a 15 km resolution. Radial velocity observations are created using the raw level-II WSR-88D data, which is dealiased, and also objectively analyzed to a 5-km resolution (Cressman 1959). Only radial velocity observations within 150 km of a particular radar that lies near or within the domain are used. Refer to Yussouf et al. (2013) and Wheatley et al. (2015) for further information on the radar data assimilation characteristics used by the WoFS.
3) Cloud water path
Cloud water path (CWP) represents the total cloud water content of a cloud at a particular point, which can be represented as a vertical summation of the hydrometeor mixing ratio values within the model (Jones et al. 2013, 2016). CWP observations from GOES-16 data are derived using the Satellite Cloud and Radiation Property retrieval System (SatCORPS, https://satcorps.larc.nasa.gov; Minnis et al. 2008a,b, 2016), which is based on the cloud property retrieval algorithms developed by Minnis et al. (2011). Data are then reanalyzed to the 5 km MRMS grid prior to assimilation. Also, a parallax correction is applied to cloudy (CWP > 0) pixels using the method described by Jones et al. (2015). Finally, clear-sky observations (CWP = 0 kg m−2) are further thinned to a 15 km resolution to prevent dry biases from developing in the system after multiple assimilation cycles. Positive CWP retrievals are only assimilated during daylight hours since the characteristics of CWP retrievals change significantly after dark. All other observation types are assimilated during the full cycling period. Further details on CWP retrievals and assimilating methods can be found in Jones et al. (2013, 2015, 2016) and Jones and Stensrud (2015).
4) Radiances
The 6.2, 6.9, and 7.3 μm infrared bands measured by GOES-16 are sensitive to upper-, mid-, and low-level atmospheric water vapor content in clear-sky regions with peak weighting functions of ~350, 450, and 625 hPa assuming a standard atmosphere. The vertical weighting can change significantly as a function of different atmospheric conditions making the assignment of vertical levels to clear-sky radiance observations challenging. In cloud regions, all three channels sense the top of the cloud layer, with colder brightness temperatures (BTs) being associated with thicker and higher altitude cloud cover.
Assimilated BT observations are obtained from the real-time L1B radiance products. All channels are sampled at a 2 km horizontal resolution at a 5 min temporal resolution for the CONUS domain. For the cloud clearing and cloud information necessary for observation processing, cloud top height from the L2 ACHAC product is combined with the L1B data and analyzed to the same 5 km grid as radar reflectivity and CWP for convenience. A parallax correction is applied for cloudy radiances since the slantwise nature of the observation between the surface and the satellite results in a displacement error in the geolocation of a cloud in satellite imagery compared to its ground truth location (e.g., Wang and Huang 2014). Without this correction, geolocation errors of up to 15 km for upper-level clouds would occur.
In clear-sky regions, we further thin the data to a 15 km resolution to reduce the impact of spatial correlation. In addition, only the 6.2 and 7.3 μm channels are assimilated in clear-sky as both have shown a high correlation with the 6.9 μm channel, strongly indicating that it would provide very little independent information to the system (Honda et al. 2018a). For cloudy regions, the full 5 km resolution observations are assimilated owing to the greater spatial variability of clouds, but only the 7.3 μm channel is retained. Since both the 6.2 and 7.3 μm channels are very highly correlated in cloudy conditions (e.g., Zhang et al. 2018), assimilating both channels would only act to assimilate cloud features twice resulting in a cloudy bias in the model after several assimilation cycles. Clear versus cloudy pixels are defined by applying the L2 cloud height product to the L1B radiance data at the corresponding time.
For both clear and cloudy radiances, the vertical level of the observation is defined using the level of the maximum Jacobian of simulated BT at each observation, which is calculated from the analysis background of each ensemble member during each assimilation cycle. For clouds, the difference between retrieved cloud top heights and the CRTM derived value was generally less than ±30 hPa. Given the large vertical localization radius being used, this difference was not considered significant.
c. Experiment configuration
Four experiment configurations are considered by this study (Table 2). The first, RADAR, assimilates conventional, radar reflectivity and radial velocity observations and acts as a control experiment to assess the overall impact of assimilating various combinations of satellite data. The second, RADCWP, assimilates GOES-16 CWP observations in addition to radar and conventional observations and closely corresponds to the spring 2019 real-time WoFS configuration. The third, CLEAR, assimilates GOES-16 clear-sky radiances in addition to CWP and radar similar to Jones et al. (2018). The final experiment, ALL, replaces positive CWP observations with all-sky BTs. Note that ALL retains CWP = 0 kg m−2 observations for cloud-clearing purposes. Also, no bias adjustments are applied to the BT observations for these experiments.
Experiment configurations evaluated by this research. Note that “Radar” refers to both reflectivity and radial velocity observations. For all-sky BT, only the 7.3 μm channel is assimilated. Both 6.2 and 7.3 μm channels are assimilated in clear-sky regions.
Several other configurations were tested, but none proved more skillful than the four described above. For radiance assimilation, both traditional (e.g., Derber and Wu 1998; Miyoshi et al. 2010; Zhu et al. 2014) and histogram-matching bias adjustment techniques were applied and tested. For clear-sky observations, application of a bias adjustment did not significantly impact the forecasts in part owing to the relatively small sample size for most cases. For all-sky observations, the biases are larger and the adjustment methods did reduce them, but the overall skill of the system when assimilating bias adjusted BTs was also lowered. This indicates that at least some of the bias being observed is model bias that needs correcting by the observations. Similar results were noted by Okamoto et al. (2019) for mesoscale data assimilation applications. Zhang et al. (2018) and Y. Zhang et al. (2019) also chose to forgo bias adjustments for storm-scale ensemble data assimilation experiments. Also, uncertainties in the representation of upper-level clouds by the cloud microphysics schemes can lead to large uncertainties in the actual observation bias present (e.g., Liu and Moncrieff 2007, Otkin and Greenwald 2008; Chaboureau and Pinty 2006).
Finally, experiments were conducted that assimilated all-sky radiances without CWP = 0 kg m−2 observations. These experiments became dominated by excessive upper-level cloud coverage in the analysis, substantially reducing overall forecast skill. At least part of this problem is due to an upper-level cloud bias in the NVD cloud microphysics scheme (Jones et al. 2018). While the configuration changes used by Jones et al. (2018) are applied here, that bias does remain to some extent, especially when no “cloud sink” observations are being assimilated.
d. Verification
This research uses the object-based verification techniques described by Skinner et al. (2016, 2018) and Jones et al. (2018) to assess the quality of 0–3 h forecasts of both radar and satellite derived objects. The object classifications applied here fall into four rough categories: precipitation (reflectivity), supercell rotation (updraft helicity), upper-level cloud coverage (infrared 11.2 μm BT), and the clear-sky environment (6.9 μm BT). For radar reflectivity and rotation objects, their definitions are similar to those used in Skinner et al. (2018). In summary, observed reflectivity objects are defined by determining locations where WoFS composite reflectivity is greater than 45 dBZ, while MRMS objects are created using the same methods, but using a matched percentile threshold to model climatology (~41 dBZ). Observed rotation objects are defined as those where 2–5 km MRMS azimuthal shear is greater than 0.004 s−1. While WoFS rotation objects are defined as those where forecast 2–5 km updraft helicity (UH) is greater than ~65 m2 s−2. In both cases, objects are generated at 5-min intervals, but rotation objects are created using a 30-min aggregation of azimuthal wind shear/updraft helicity centered on the valid forecast time, for each ensemble member over the duration of the forecast period.
The procedure for BT112 objects closely follows the infrared object classification method used by Jones et al. (2018) and also builds on work by Griffin et al. (2017a,b). Simulated GOES-16 satellite data are generated for all ensemble members using version 2.3 of the CRTM. BT112 objects are defined as those where observations and simulated BT112 are less than 216 K, respectively. These thresholds were selected to primarily emphasize the locations of strong convection, though some upper-level cirrus of nonconvective origin can also reach this threshold. For this work, a second satellite derived object classification is used where areas of dry air are defined by warm water vapor channel BTs to quantify the impact of assimilating satellite observations on the near-storm environment. We used the midlevel water vapor channel (BT69) to generate these objects with thresholds of 250 K for both observations and WoFS forecasts, respectively. These objects, labeled “dry-air objects” will be used to validate midlevel moisture characteristics.
Forecast and observed objects are matched in time and space using the total interest score (Davis et al. 2006) defined in Skinner et al. (2018). Object matching allows matched forecast objects, unmatched forecast objects, and unmatched observed objects to be classified as “hits,” “false alarms,” and “misses,” respectively, and contingency table–based metrics to be used to quantify the forecast skill. Compared to radar objects, satellite objects are generally much larger, but fewer in number. Thus, the search radii required for object matching is much larger, up to 400 km compared to 40 km for radar data objects (Jones et al. 2018). Observed and forecast BT objects are generated at 10 min intervals for the duration of the forecast period. All object-based verification is computed using a 270 × 270 gridpoint domain to remove edge artifacts from the objective comparison.
3. Event overviews
Three severe weather events occurring in May 2019 and one event in July 2019 were selected to analyze the impacts of various combinations of satellite data assimilation in the WoFS. All cases generated multiple instances of high-impact weather including tornadoes in addition to large hail and damaging straight-line winds. Outside this common link, the atmospheric characteristics of each varied substantially from case to case.
Two long-track supercells developed on 17 May with one located in southwest Nebraska (NE) between 2330 and 0130 UTC and the other located in southwest Kansas (KS) between 0030 and 0430 UTC. Composite radar reflectivity at 0100 18 May shows both supercells along with nontornadic convection in central and northern NE (Fig. 1a). Corresponding GOES-16 BT69 is shown in Fig. 1b. At this time, the NE storm has generated multiple tornadoes and is associated with higher reflectivity values and a larger area of cold cloud tops compared to the KS storm, which is still developing. Satellite data also show a north–south oriented band of upper-level cirrus clouds overrunning the region where the KS storm develops (Fig. 1b).
The 22 May case consisted of a stationary cold front extending from northeastern OK into central Missouri (MO) with an environment favorable for tornadic supercells existing south of this boundary. By 2300 UTC several supercells were present in northeastern OK already having generated several tornado reports (Fig. 1c). Additional severe convection was present in north-central OK and northern Texas (TX). Many of these storms are in close proximity to each other, increasing the difficulty of forecasting individual severe weather tracks. Satellite data also indicated developing convection along the front in central MO, which proceeded to generate multiple severe weather reports, including tornadoes after 0030 UTC (Fig. 1d).
The 28 May case contained two tornadic supercells in central and eastern KS with a complex of severe convection located in northern MO and Iowa (IA) and central OK by 2300 UTC (Fig. 1e). Of interest was the large difference in the satellite presentation of the two tornadic storms. Both generated tornadoes by 2300 UTC, but the western KS storm has a very small cirrus shield compared to the much larger and colder one associated with the eastern KS storm, which also merges with the severe convection to the northeast (Fig. 1f). The eastern KS supercell generated a violent tornado at 2340 UTC near Linwood, KS, and threatened the Kansas City metropolitan area, but fortunately weakened just prior to entering this area.
The 19 July case differed from the other three cases in being a severe wind threat rather than a long-track tornado threat in the northern plains. By 0000 UTC 20 July, a large complex of convection was rapidly moving southeast through Wisconsin (WI) having already generated numerous severe wind and a few tornado reports in its wake (Fig. 1g). Satellite observations indicated a well-developed cirrus shield propagating north and east of the convection while also showing a relatively clear-sky environment ahead of the convection (Fig. 1h).
4. Assimilation statistics
The innovation (or bias) and root-mean-square innovation (RMSI) are calculated for clear-sky 6.2 and 7.3 μm clear-sky BT observations (BT62clear, BT73clear) from the CLEAR experiment and 7.3 μm all-sky BT observations (BT73all) from ALL for each event. For this work, innovation is defined as the observation minus the ensemble mean prior (forecast) or posterior (analysis). The number of observations assimilated during each cycle are shown in Fig. 2 for BT62clear and BT73all. The sample size for BT73clear is very similar to the BT62clear sample size (not shown). The number of BT62clear observations assimilated varies as a function of the amount of cloud cover present within each domain. For 17 and 22 May, the number gradually decreases as a function of time through ~2200 UTC as convection develops and matures. Afterward, some convection moves outside the domain and the number of clear-sky observations increases again. The 28 May case differs due to the large amount of cloud cover present early in the assimilation period, which leaves the domain after 2100 UTC, while new convection develops after 0000 UTC reducing clear-sky observations again. The 19 July case assimilates the greatest number of clear-sky observations early in the assimilation cycle, decreasing at later times as cloud cover associated with the MCS covers larger areas of the domain. The number of BT73all observations assimilated is greater compared to clear-sky observations ranging from 7500 to 9500 for all cases after the initial spin up period.
Prior bias for BT62clear observations ranges from 0 to 1 K (observations are warmer than the model) (Fig. 3a). Overall average biases are 0.53, 0.16, 0.67, and 0.29 K for 17, 22, 28 May, and 19 July, respectively (Table 3). Post assimilation biases are also generally small, being less than 0.5 K (Fig. 3a, Table 3). BT73clear biases are similar except that the magnitude of the biases is somewhat smaller (Fig. 3c, Table 3). BT73all observation bias differs from the clear-sky bias in several ways (Fig. 3e). First, the values are generally negative, indicating that observations are generally colder than the model. For 17 and 28 May, prior biases are on the order of −0.5 K or less, decreasing to less than −0.25 K out to ~2200 UTC. Afterward, biases increase somewhat due to the increase in convective cirrus over the domain. Prior and posterior bias for the 22 May and 19 July cases are also quite small out to 0000 UTC though biases do increase somewhat thereafter as the upper-level cloud coverage encompasses more of the model domain. Overall, average prior biases range between −0.7 and −1.5 K for all case with posterior biases on the order of −0.2 (Table 3).
Prior and posterior biases and RMSI averaged overall assimilation cycles for each case for clear-sky 6.2 and 7.3 μm observations from CLEAR and 7.3 μm all-sky observations from ALL.
RMSI for clear- and all-sky radiances generally shows the same pattern as the biases for each case. For BT62clear and BT73clear prior RMSI is less than 1.5 K for both the 17 and 28 May cases (Figs. 3b,d). The prior RMSI is quite small for the 22 May and 19 July cases and remains less than 0.8 K at all analysis times. The corresponding posterior RMSI is generally below 1.0 K for all cases with 19 July again generating the lowest errors (Figs. 3b,d, Table 3). The RMSI for BT73all is larger, as would be expected with prior values ranging between 4 and 5 K (Fig. 3f, Table 3). After assimilation, RMSI generally decreases to 1.5 K or less.
Innovation and RMSI statistics for radar reflectivity, radial velocity, and satellite retrieved CWP observations are all similar to those described by Wheatley et al. (2015) and Jones et al. (2016). As the results are very similar, the corresponding figures from these references are not reproduced here.
5. Experiment comparisons
a. Examples
1) 17 May
By 2300 UTC 17 May, the NE supercell was well established and had already produced several tornadoes and would continue to do so until 0030 UTC. Convection farther south had yet to develop and would not generate its first tornado report until 0030 UTC 18 May. To assess the impact of assimilating satellite data on the forecast of both storms, reflectivity, UH, synthetic satellite, and environmental forecasts are generated for a 0–3 h period starting at 2300 UTC. The 3-h composite reflectivity forecast valid at 0200 UTC shows that several important differences exist between each experiment (Fig. 4). Many ensemble members forecast reflectivity greater than 45 dBZ near the observed location of the NE supercell, but the same members often overforecast storm coverage farther north (Fig. 4a). Both RADCWP and CLEAR generate slightly less coverage in northern NE, but appear to move the NE supercell northeast too fast (Figs. 4b,c). ALL generates slower storm motion than RADAR while limiting the false alarms farther north (Fig. 4d). The biggest difference between each experiment is the treatment of the southern KS convection. Only ALL forecasts convection in this location by 0100 UTC for a majority of the members; however, the coverage does exceed observations to some degree (Fig. 4d).
Transitioning to 0–3 h forecasts of 2–5 km UH greater than 60 m2 s−2, we find that all experiments generate a long swath of greater than 50% probabilities associated with the NE storm, which corresponds well with the reported tornadoes during this period and the tornado warning present valid at 0200 UTC (Fig. 5). However, another high-probability swath exists from northeastern CO into southern NE that was not associated with any tornado reports, but did generate a few hail reports. Note that ALL generates the lowest UH probabilities associated with this storm (Fig. 5d). ALL also forecasts higher UH probabilities associated with the tornadic NE storm between 2300 and 0000 UTC compared to the other experiments. With respect to the KS storm, ALL generates two UH swaths associated with two developing storms. The northern track of the two lines up well with corresponding hail reports and tornado warnings valid at 0200 UTC, but the storm motion in the model is too fast (Figs. 4d and 5d). Similarly, a second track is forecast farther south, which does not line up with observations. Later forecast start times closer to convective initiation correct these spatial and temporal displacement errors.
It is important to understand why this improvement to the prediction of the KS storms occurs in ALL, which requires an assessment of the impacts of assimilating all-sky radiances to the overall environment. The ensemble mean BT69 3 h forecast valid at 0200 UTC 18 May shows several key differences in the midlevel moisture environment and upper-level clouds (Fig. 6). RADAR, RADCWP, and CLEAR fail to generate convective clouds in KS at this time while also forecasting an area of relatively dry air present over western KS (Figs. 6a–c). Conversely, ALL forecasts contain a moister environment both behind and ahead of the developing convection in KS (Fig. 6d). In both cases, this forecast represents a better match to the observed satellite observations at this time, although the coverage of the KS convection is overforecast somewhat (Fig. 6e). To further assess differences in the forecast environment, 30 min forecasts of ensemble mean surface temperature and dewpoint valid at 2330 UTC are analyzed (Fig. 7). The impact of increased cirrus cloud coverage is evident along the KS–CO border with ALL generating colder temperatures compared to other experiments (Figs. 7a,b). Also, the dryline is positioned farther west in ALL, nearer to the location of observed convective initiation. As a result, ALL is able to sustain analyzed convection in this region in the more favorable environment compared to the other experiments. Note that CLEAR and RADAR are similar to RADCWP in this respect (not shown). Bias (forecast–observations) and root-mean-square error (RMSE) for the 78 surface observations in the domain show that ALL generates a small cold and moist bias compared to RADCWP, but the dewpoint error is reduced.
2) 22 May
Several differences between each experiment are evident from 3 h forecasts of reflectivity initiated at 2200 UTC (Fig. 8). During this period, convection develops and becomes severe in northeastern OK and also begins to develop farther north along the front during the last hour of the forecast period. RADAR appears to overforecast reflectivity coverage in northeastern OK by 0100 UTC compared to those that assimilate satellite data. However, the other experiments place the tornadic supercell too far east (Figs. 8b–d). Differences between the satellite data assimilation experiments are smaller. CLEAR performs poorly with the northern OK convection with no ensemble members forecasting reflectivity greater than 45 dBZ in this region. In ALL, some members do forecast this convection and also accurately forecast convection moving into southern OK. However, ALL also has the fewest members forecasting the northeastern OK supercell compared to the other experiments (Fig. 7).
Corresponding 2–5 km UH forecasts also differ significantly between each experiment (Fig. 9). Experiments assimilating positive CWP data (RADCWP and CLEAR) perform significantly better with the severe convection in central and eastern MO with high probability UH swaths better matching the location of severe weather reports and warnings during this time. For the northeastern OK supercell, these experiments forecast a narrower UH swath than RADAR, resulting in displacement error between warnings and forecasts by 0100 UTC. ALL differs from the other experiments in several other ways (Fig. 9d). First, it forecasts high probability UH swaths associated with tornado warning in central MO whereas none of the other experiments correctly forecast this storm. In OK, ALL weakens the primary supercell much quicker compared than the other experiments, but this also corresponds with the lack of tornado reports associated with this storm between 0000 and 0100 UTC. Finally, over half of the ensemble members in ALL correctly forecast a tornadic storm in far north TX which is mostly missed by the other experiments.
Forecast differences also extend to the cloud fields, as shown by ensemble mean simulated BT69 at 0100 UTC (Fig. 10). None of the experiments accurately forecast the westward extent of cold cloud tops associated with the northeastern OK supercell and also poorly forecast the central OK cloud tops, consistent with the reflectivity forecasts described above. Assimilating CWP data warms cloud top temperatures compared to RADAR. ALL restores the colder cloud tops, but does not correct the location errors observed in northern OK. ALL does have an improved representation of the southern OK cloud cover compared to the other experiments. Differences in the surface conditions are generally small for this and the following cases (not shown).
3) 28 May
Forecasts for 28 May also show important differences between each experiment. 3-h reflectivity forecasts initiated at 2100 UTC indicate that all experiments forecast a large area of convection in northern MO and southern IA with more isolated convection in KS and OK, which is generally consistent with observations (Fig. 11). There are three individual areas of interest at this time. The first is a tornado-producing storm in north-central KS, the second is another tornado-producing storm in eastern KS, and finally a severe bow echo in southeastern IA. RADAR forecasts both the central KS and IA convection well, but only a few ensemble members predict the eastern KS storm (Fig. 11a). RADAR also overforecasts convection in many areas. RADCWP and CLEAR are similar, but the overforecasting of convection appears less significant, though CLEAR does generate more convection in northern OK compared to the other two (Fig. 11c). ALL shows more significant differences for all three areas of interest (Fig. 11d). First, it is somewhat too slow with the central KS storm, but it does correctly forecast the eastern KS storm. Finally, it weakens the eastern IA convection too fast as it moves into IL.
Comparing 3-h UH probability swaths further emphasizes the differences between each experiment (Fig. 12). While the reflectivity forecast for the central KS storm from RADAR is accurate, it generates the lowest UH probabilities among the four experiments, indicating the overall organization of this storm in RADAR is poor (Fig. 12a). UH probabilities associated with the severe wind threat in eastern IA and western IL are lower than either RADCWP or CLEAR. However, RADAR does forecast high UH probabilities associated with the eastern KS storm early in the forecast period, but they decrease to near zero by the time of the first tornado report. ALL generated a more accurate prediction of rotation in the eastern KS storm than other experiments. The quality of the ALL forecast of the eastern KS supercell is noteworthy as the forecast was issued over 2 h before genesis of a long-track, violent tornado that impacted a major metropolitan area. Despite the accurate prediction of the eastern KS supercell, ALL forecast later arrival times for the central KS storms compared with severe weather reports and warnings (Fig. 12d). Additionally, as with RADAR, ALL generated lower UH probabilities for the eastern IA convection due to storm dissipation 2 h into the forecast.
The 3-h ensemble-mean-simulated BT69 valid at 0000 UTC shows several key differences in the forecast cloud characteristics of each experiment. First, the convective cirrus associated with the central KS storm is much smaller and warmer compared to the other storms. This feature is forecast by every experiment, but RADAR generates the smallest coverage of BT69 < 215 K (Fig. 13). Convective cirrus for the remainder of the storms is generally larger and produces colder cloud tops. ALL correctly extends these cold cloud tops farther into northeastern KS, corresponding to the long-track supercell, whereas the other experiments do not have this feature.
4) 19 July
The forecast impacts from assimilating satellite data are clearly evident in the July 19 case. Figure 14 shows 3-h composite reflectivity forecasts initiated at 2300 UTC. All experiments correctly forecast a southwest–northeast-oriented convective complex moving southeast. RADAR is the outlier and forecasts storm motion too slow compared to observations (Fig. 14a). Assimilating satellite data in one form or another increases the propagation speed, generating more skillful reflectivity forecasts for at least the northern half of the convective complex (Figs. 14b–d). RADCWP and CLEAR generated spurious convection in far northern WI, which is not present in either RADAR or ALL. Qualitatively, RADCWP performs best while all satellite DA experiments show some improvement over assimilating radar data only. Since the primary convective hazard of this event was severe straight-line winds rather than tornadoes, the 3-h forecast probability of surface wind gusts greater than 50 kt (1 kt ≈ 0.51 m s−1) are shown in place of the UH forecasts provided for other cases. All experiments generate a large swath of modest-to-high severe wind probabilities moving southeastward, generally matching severe weather reports and warnings during this period (Fig. 15). Assimilating satellite data causes two important changes. First, these experiments isolate the northern portion of the complex early and allow it to develop more rapidly in the model. Second, the probabilities of severe wind along the southern edge of the convection are lower than generated by RADAR, especially for the ALL experiment (Fig. 15d).
Differences in the thermodynamic environment of experiments that assimilate satellite observations can be inferred from the 3-h forecasts of simulated BT69. Assimilating satellite data increases BT69 ahead of the convection compared to RADAR, which indicates a dryer midtroposphere (Fig. 16). Comparing the forecast to observations indicates that this drying may be overdone (Fig. 16e). The characteristics of the convection itself are very similar across all experiments, though forecast cloud top BT69 are somewhat colder in the forecast compared to the observed values.
b. Statistics and performance
While the qualitative comparisons of example forecasts from each case show significant differences due to assimilating various combinations of radar and satellite data, it is important to quantify these differences and determine the relative skill of each experiment. To show these differences, performance diagrams (Roebber 2009) relating probability of detection (POD), false alarm ratio (FAR), critical success index (CSI), and frequency bias for composite reflectivity, rotation (2–5 km UH), 6.9, and 11.2 μm objects, respectively (Skinner et al. 2018; Jones et al. 2018). Figure 17 shows 1, 2, and 3 h forecast performance for each of these parameters calculated overall forecasts from the 17 May case. The overall quality of reflectivity and rotation forecasts from all experiments is good, as evidenced by 1-h-forecast CSI values exceeding 0.45 for reflectivity and 0.4 for UH before decreasing somewhat at later forecast times (Figs. 17a–f). ALL generates somewhat lower skill than RADAR or the CWP experiments out to approximately 2 h, but performs better relative to the other experiments by the end of the forecast period. For both reflectivity and UH, the larger increase in FAR relative to POD is the reason for the slight decrease in skill in the ALL experiment, which reverses by the 3-h forecast when ALL generates a much higher POD with little increase in FAR. This evolution is consistent with the example shown above where longer-term forecasts of the southern KS storms were predicted by ALL, but at the cost of some additional false alarms. To assess why the 2–3 h forecast skill for reflectivity and UH is superior in the ALL experiment, the skill of the environment (i.e., dry-air objects) and upper-level clouds are also assessed. ALL demonstrates much improved skill of the cloud-free environment at all forecast times, and this improvement likely translates into improved longer-term forecasts of convection as the relative impact of the mesoscale environment becomes more important than storm-scale initial conditions (Figs. 17g–i). For upper-level clouds, the performance characteristics are similar to reflectivity, with ALL generating increased false alarms in the early forecast period, but maintaining a greater POD later in the forecast period (Figs. 17j–l). RADCWP, CLEAR, and to a lesser extent ALL, generally outperform RADAR for both reflectivity and UH at all forecast times, indicating that assimilating satellite data in any form improves model skill.
Overall reflectivity and UH forecast skill remains high for the 22 May case in all experiments. Reflectivity CSI ranges from greater than 0.5 at 1 h to approximately 0.4 at 3 h (Figs. 18a–c). CSI is similar for all experiments, but RADCWP and CLEAR are relatively unbiased in the early forecast period whereas RADAR and ALL have a noticeable positive bias, with RADAR being the most biased. This is consistent the spurious reflectivity forecasts from RADAR shown in the example described above (Figs. 8a,d). UH forecasts at 1 h also show a similar pattern, but RADAR and ALL perform better than RADCWP and CLEAR by the 2 and 3 h forecast periods (Figs. 18d–f). This evolution indicates that assimilating CWP modifies the environment to inhibit development of rotating storms as the forecast time increases. Note that reflectivity skill does not show this difference. From the environmental perspective only a few, large dry-air objects were defined; thus, overall skill was very good for all experiments (Figs. 18g–i). Differences in false alarms are due to ALL generating 2–3 extra objects compared to the other experiments. Finally, ALL performs much better with upper-level cirrus out to 2 h, primarily through the improved forecast of the southern OK convection (Figs. 18j–l).
On 28 May, the differences between each experiment are generally small for all forecast parameters, but also remain mostly stable throughout the 3 h forecast period with only small decreases as a function of time (Fig. 19). For reflectivity and UH, CLEAR generally performs best followed by RADCWP and ALL, with RADAR having the lowest CSI. Satellite data assimilation also improves forecasts of dry-air and upper-level cloud objects to some extent, with CLEAR again being the best performer overall (Figs. 19g–l). These statistics do not reflect the improved prediction of the eastern KS supercell in ALL as the large number of nontornadic storms in northern MO masks the contribution from higher-impact events. This masking of high-impact events is a limitation of bulk verification measures and illustrates the importance of complementing them with subjective analyses.
In the 19 July case, the impact of assimilating satellite observations is very evident in reflectivity forecast skill. Due to the faster motion of convection in these experiments, FAR is significantly reduced, with the magnitude of the difference increasing at later forecast times (Figs. 20a–c). For UH, satellite data assimilation clearly improves skill in the early forecast period, but this difference decreases at later forecast times (Figs. 20d–f). Note that object-based verification of wind-gust forecasts is not possible at this time owing to lack of an acceptable verification dataset. The number of satellite objects is generally small since this case was characterized by one large area of convection and another large area of cloud-free conditions for most of its duration. Still, some differences are evident. Satellite data assimilation improved dry-air object skill for the 0–2 h time period (Figs. 20g,h), while it performed worse than RADAR for upper-level cloud forecasts (Figs. 20j–l). As with other cases, there appears to be a positive upper-level cloud bias generated in the model after repeated assimilation of these cloud features. Assimilation of all-sky radiances in particular worsens this bias. Future research efforts will focus on methods to reduce these cloud biases introduced through satellite data assimilation while maintaining the many positive elements of assimilating these data.
To assess overall performance, skill scores are calculated across all experiments over each forecast period for object types. Figure 21 indicates the best and worst experiment defined by ensemble mean CSI at 30 min forecast intervals out to 3 h (180 min) for each variable. RADAR is generally the poorest performer for all object types with the exception of 30–60 min BT69 and 180 min BT112 forecasts. Satellite data assimilation experiments perform well across all object types with CLEAR performing best for reflectivity out to 90 min and BT112 at all forecast times while ALL performs best for BT69 at all forecast times and reflectivity forecasts after 120 min. Differences in rotation forecasts are generally smaller, with both CLEAR and RADCWP generating similar values and trading the highest skill out to 120 min. At later forecast times, the difference in CSI between the worst and best performing model is less than 0.015.
6. Conclusions
Assimilating satellite data into the WoFS clearly benefited high-impact weather forecasts compared to only assimilating radar data, which is consistent with previous findings (e.g., Jones et al. 2016). There were important forecast differences depending on which satellite data type was assimilated. Assimilating CWP generally improved forecasts of reflectivity and rotation compared to radar-only experiments, but has smaller impacts to the near-storm environment and upper-level cloud forecasts. For the May cases, the number of clear-sky radiance observations was relatively small limiting their impact in most instances. Many more clear-sky radiance observations were assimilated on 19 July, leading to larger improvements to forecast skill.
Assimilating all-sky radiances generally had large impacts on the forecasts compared against radar-only or retrieval assimilation techniques. Radiance assimilation generally improved convective initiation forecasts, as shown by the 17 and 28 May cases, with secondary improvements in the near-storm environment surrounding ongoing convection. However, some negative aspects to all-sky radiance assimilation were also observed. The most significant was an upper-level cloud bias as assimilating cirrus clouds became too expansive and too thick. This led to negative impacts to the thermodynamic environment resulting in a degradation of forecasts later in some cases. Qualitatively, the retrieval method combined with clear-sky radiances generated the best forecast skill of high-impact weather prediction for all object types except BT69, but this version of the system also benefited from several years of tuning.
This research only represents a first step at all-sky radiance assimilation into the WoFS. Many refinements will be required so that the advantages of all-sky radiances DA can be retained while removing the unwanted side effects produced in the cases studied here. Ongoing research will focus on several key aspects of the system. Further enhancements to the cloud microphysics scheme are likely to better handle a rapid update of nonconvective clouds. Also, improvements to the model itself and evaluations of optimal horizontal and vertical resolutions for satellite data assimilation are being performed. Research on adaptive thinning of radar and satellite data are also under way as it is likely that the amount of data from each sensor currently being assimilated is not optimal. Future versions of the WoFS will utilize methods that compare observations with ensemble spread and to each other to determine which observations will be most effective to assimilate. Observations that contain duplicate and/or conflicting information will not be assimilated. Future systems are also expected to use a cloud classification algorithm to define the observation error, data density, and localization radius to apply to various radiance and/or CWP observations prior to assimilation. Currently, all cloud types are treated equally and the results shown here clearly indicate that the characteristics of cirrus clouds, low-level clouds, and those associated with convective initiation should be assessed in a rigorous manner. Combined, these enhancements should resolve some of the biases observed in this research.
Acknowledgments
This research was funded in part by the NOAA Warn-on-Forecast project. Additional funding was provided by NASA ROSES NNX15AR57G, NOAA/OAR/OWAQ FY2016 Joint Technology Transfer Initiative Grants NA16OAR4590242, NA16OAR4320115, and under the NOAA–University of Oklahoma Cooperative Agreement NA16OAR4320115. Support for WoF computing resources was provided by Gerry Creager. HRRRE initial and boundary conditions for this work were provided by the Earth System Research Laboratory, Global Systems Division as part of real-time experiments in 2019 with the aid of David Dowell, Therese Ladwig, and Curtis Alexander. Kristopher Bedka, William Smith Jr., and Rabindra Palikonda are also supported by the NASA MAP Program.
REFERENCES
Aksoy, A., D. Dowell, and C. Snyder, 2009: A multicase comparative assessment of the ensemble Kalman filter for assimilation of radar observations. Part I: Storm-scale analyses. Mon. Wea. Rev., 137, 1805–1824, https://doi.org/10.1175/2008MWR2691.1.
Aksoy, A., D. Dowell, and C. Snyder, 2010: A multicase comparative assessment of the ensemble Kalman filter for assimilation of radar observations. Part II: Short-range ensemble forecasts. Mon. Wea. Rev., 138, 1273–1292, https://doi.org/10.1175/2009MWR3086.1.
Alexander, C. R., and Coauthors, 2018: Development of the High-Resolution Rapid Refresh Ensemble (HRRRE). 22nd Conf. on Integrated Observing and Assimilation Systems for the Atmosphere, Oceans, and Land Surface (IOAS-AOLS), Austin, TX, Amer. Meteor. Soc., 11.3, https://ams.confex.com/ams/98Annual/webprogram/Paper335526.html.
Anderson, J. L., 2009: Spatially and temporally varying adaptive covariance inflation for ensemble filters. Tellus, 61A, 72–83, https://doi.org/10.1111/j.1600-0870.2008.00361.x.
Auligné, T., A. Lorenc, Y. Michel, T. Montmerle, A. Jones, M. Hu, and J. Dudhia, 2011: Toward a new cloud analysis and prediction system. Bull. Amer. Meteor. Soc., 92, 207–210, https://doi.org/10.1175/2010BAMS2978.1.
Benjamin, S. G., and Coauthors, 2016: A North American hourly assimilation and model forecast cycle: The Rapid Refresh. Mon. Wea. Rev., 144, 1669–1694, https://doi.org/10.1175/MWR-D-15-0242.1.
Chaboureau, J.-P., and J.-P. Pinty, 2006: Validation of a cirrus parameterization with Meteosat Second Generation observations. Geophys. Res. Lett., 33, L03815, https://doi.org/10.1029/2005GL024725.
Choate, J. J., A. J. Clark, P. L. Heinselman, D. A. Imy, and P. S. Skinner, 2018: First demonstration of the NSSL experimental Warn-on-Forecast system as part of the 2017 spring experiment. Eighth Conf. on Transition of Research to Operations, Austin, TX, Amer. Meteor. Soc., 1194, https://ams.confex.com/ams/98Annual/webprogram/Paper335289.html.
Cressman, G. P., 1959: An operational objective analysis system. Mon. Wea. Rev., 87, 367–374, https://doi.org/10.1175/1520-0493(1959)087<0367:AOOAS>2.0.CO;2.
Davis, C., B. Brown, and R. Bullock, 2006: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Wea. Rev., 134, 1772–1784, https://doi.org/10.1175/MWR3145.1.
Derber, J. C., and W.-S. Wu, 1998: The use of TOVS cloud-cleared radiances in the NCEP SSI analysis system. Mon. Wea. Rev., 126, 2287–2299, https://doi.org/10.1175/1520-0493(1998)126<2287:TUOTCC>2.0.CO;2.
Dowell, D., L. J. Wicker, and C. Snyder, 2011: Ensemble Kalman filter assimilation of radar observations of the 8 May 2003 Oklahoma City supercell: Influences of reflectivity observations on storm-scale analyses. Mon. Wea. Rev., 139, 272–294, https://doi.org/10.1175/2010MWR3438.1.
Gallo, B. T., and Coauthors, 2017: Breaking new ground in severe weather prediction: The 2015 NOAA/Hazardous Weather Testbed Spring Forecasting Experiment. Wea. Forecasting, 32, 1541–1568, https://doi.org/10.1175/WAF-D-16-0178.1.
Gaspari, G., and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteor. Soc., 125, 723–757, https://doi.org/10.1002/qj.49712555417.
Griffin, S. M., J. A. Otkin, C. M. Rozoff, J. M. Sieglaff, L. M. Cronce, C. R. Alexander, T. L. Jensen, and J. K. Wolff, 2017a: Seasonal analysis of cloud objects in the High-Resolution Rapid Refresh (HRRR) model using object-based verification. J. Appl. Meteor. Climatol., 56, 2317–2334, https://doi.org/10.1175/JAMC-D-17-0004.1.
Griffin, S. M., J. A. Otkin, C. M. Rozoff, J. M. Sieglaff, L. M. Cronce, and C. R. Alexander, 2017b: Methods for comparing simulated and observed satellite infrared brightness temperatures and what do they tell us? Wea. Forecasting, 32, 5–25, https://doi.org/10.1175/WAF-D-16-0098.1.
Han, Y., F. Weng, Q. Liu, and P. van Delst, 2007: A fast radiative transfer model for SSMIS upper atmosphere sounding channels. J. Geophys. Res., 112, D11121, https://doi.org/10.1029/2006JD008208.
Honda, T., and Coauthors, 2018a: Assimilating all-sky Himawari-8 infrared radiances: A case of Typhoon Soudelor (2015). Mon. Wea. Rev., 146, 213–229, https://doi.org/10.1175/MWR-D-16-0357.1.
Honda, T., S. Kotsuki, G.-Y. Lien, Y. Maejima, K. Okamoto, and T. Miyoshi, 2018b: Assimilation of Himawari-8 all-sky radiances every 10 minutes: Impact on precipitation and flood risk prediction. J. Geophys. Res. Atmos., 123, 965–976, https://doi.org/10.1002/2017JD027096.
Hu, J., N. Yussouf, D. Turner, T. A. Jones, and X. Wang, 2019: Impact of ground-based remote sensing boundary layer observations on short-term probabilistic forecasts of a tornadic supercell event. Wea. Forecasting, 34, 1453–1476, https://doi.org/10.1175/WAF-D-18-0200.1.
Hu, M., H. Shao, D. Stark, K. Newman, C. Zhou, and X. Zhang, 2016: Gridpoint Statistical Interpolation (GSI) user’s guide version 3.5. Developmental Testbed Center Rep., 141 pp., https://dtcenter.org/com-GSI/users/docs/users_guide/GSIUserGuide_v3.5.pdf.
Johnson, A., X. Wang, J. Carely, L. Wicker, and C. Karstens, 2015: A comparison of multiscale GSI-based EnKF and 3DVar data assimilation using radar and conventional observations for mid-latitude convective-scale precipitation forecasts. Mon. Wea. Rev., 143, 3087–3108, https://doi.org/10.1175/MWR-D-14-00345.1.
Jones, T. A., and D. J. Stensrud, 2015: Assimilating cloud water path as a function of model cloud microphysics in an idealized simulation. Mon. Wea. Rev., 143, 2052–2081, https://doi.org/10.1175/MWR-D-14-00266.1.
Jones, T. A., J. A. Otkin, D. J. Stensrud, and K. Knopfmeier, 2013: Assimilation of satellite infrared radiances and Doppler radar observations during a cool season observing system simulation experiment. Mon. Wea. Rev., 141, 3273–3299, https://doi.org/10.1175/MWR-D-12-00267.1.
Jones, T. A., J. A. Otkin, D. J. Stensrud, and K. Knopfmeier, 2014: Forecast evaluation of an observing system simulation experiment assimilating both radar and satellite data. Mon. Wea. Rev., 142, 107–124, https://doi.org/10.1175/MWR-D-13-00151.1.
Jones, T. A., D. Stensrud, L. Wicker, P. Minnis, and R. Palikonda, 2015: Simultaneous radar and satellite data storm-scale assimilation using an ensemble Kalman filter approach for 24 May 2011. Mon. Wea. Rev., 143, 165–194, https://doi.org/10.1175/MWR-D-14-00180.1.
Jones, T. A., K. Knopfmeier, D. Wheatley, G. Creager, P. Minnis, and R. Palikonda, 2016: Storm- scale data assimilation and ensemble forecasting with the NSSL experimental Warn-on- Forecast system. Part II: Combined radar and satellite data experiments. Wea. Forecasting, 31, 297–327, https://doi.org/10.1175/WAF-D-15-0107.1.
Jones, T. A., X. Wang, P. Skinner, A. Johnson, and Y. Wang, 2018: Assimilation of GOES-13 imager clear-sky water vapor (6.5 μm) radiances into a Warn-on-Forecast system. Mon. Wea. Rev., 146, 1077–1107, https://doi.org/10.1175/MWR-D-17-0280.1.
Kleist, D. T., D. F. Parrish, J. C. Derber, R. Treadon, W.-S. Wu, and S. Lord, 2009: Introduction of the GSI into the NCEP Global Data Assimilation System. Wea. Forecasting, 24, 1691–1705, https://doi.org/10.1175/2009WAF2222201.1.
Liu, C., and M. W. Moncrieff, 2007: Sensitivity of cloud resolving simulations of warm-season convection to cloud microphysics parameterizations. Mon. Wea. Rev., 135, 2854–2868, https://doi.org/10.1175/MWR3437.1.
Mansell, E. R., C. L. Ziegler, and E. C. Bruning, 2010: Simulated electrification of a small thunderstorm with two-moment bulk microphysics. J. Atmos. Sci., 67, 171–194, https://doi.org/10.1175/2009JAS2965.1.
McNally, A. P., J. C. Derber, W. Wu, and B. B. Katz, 2000: The use of TOVS level-1b radiances in the NCEP SSI analysis system. Quart. J. Roy. Meteor. Soc., 126, 689–724, https://doi.org/10.1002/qj.49712656315.
McNally, A. P., P. D. Watts, J. A. Smith, R. Engelen, G. A. Kelly, J. N. Thepaut, and M. Matricardi, 2006: The assimilation of AIRS radiance data at ECMWF. Quart. J. Roy. Meteor. Soc., 132, 935–957, https://doi.org/10.1256/qj.04.171.
McPherson, R. A., and Coauthors, 2007: Statewide monitoring of the mesoscale environment: A technical update on the Oklahoma Mesonet. J. Atmos. Oceanic Technol., 24, 301–321, https://doi.org/10.1175/JTECH1976.1.
Migliorini, S., 2012: On the equivalence between radiance and retrieval assimilation. Mon. Wea. Rev., 140, 258–265, https://doi.org/10.1175/MWR-D-10-05047.1.
Minamide, M., and F. Zhang, 2019: An adaptive background error inflation method for assimilating all-sky radiances. Quart. J. Roy. Meteor. Soc., 145, 805–823, https://doi.org/10.1002/qj.3466.
Minnis, P., C. R. Yost, S. Sun-Mack, and Y. Chen, 2008a: Estimating the top altitude of optically thick ice clouds from thermal infrared satellite observations using CALIPSO data. Geophys. Res. Lett., 35, L12801, https://doi.org/10.1029/2008GL033947.
Minnis, P., and Coauthors, 2008b: Near-real time cloud retrievals from operational and research meteorological satellites. Proc. SPIE, 7107, 710703, https://doi.org/10.1117/12.800344.
Minnis, P., and Coauthors, 2011: CERES Edition-2 cloud property retrievals using TRMM VIRS and Terra and Aqua MODIS data—Part I: Algorithms. IEEE Trans. Geosci. Remote Sens., 49, 4374–4400, https://doi.org/10.1109/TGRS.2011.2144601.
Minnis, P., and Coauthors, 2016: A consistent long-term cloud and clear-sky radiation property dataset from the Advanced Very High Resolution Radiometer (AVHRR). Algorithm Theoretical Basis Doc. CDRP-ATBD-0826, 159 pp.
Miyoshi, T., Y. Sato, and T. Kadowaki, 2010: Ensemble Kalman filter and 4D-Var intercomparison with the Japanese operational global analysis and prediction system. Mon. Wea. Rev., 138, 2846–2866, https://doi.org/10.1175/2010MWR3209.1.
Okamoto, K., Y. Sawada, and M. Kunii, 2019: Comparison of assimilating all-sky and clear-sky infrared radiances from Himawari-8 in a mesoscale system. Quart. J. Roy. Meteor. Soc., 145, 745–766, https://doi.org/10.1002/qj.3463.
Otkin, J., 2012a: Assimilation of water vapor sensitive infrared brightness temperature observations during a high impact weather event. J. Geophys. Res., 117, D19203, https://doi.org/10.1029/2012JD017568.
Otkin, J., 2012b: Assessing the impact of the covariance localization radius when assimilating infrared brightness temperature observations using an ensemble Kalman filter. Mon. Wea. Rev., 140, 543–561, https://doi.org/10.1175/MWR-D-11-00084.1.
Otkin, J., and T. J. Greenwald, 2008: Comparison of WRF model-simulated MODIS-derived cloud data. Mon. Wea. Rev., 136, 1957–1970, https://doi.org/10.1175/2007MWR2293.1.
Polkinghorne, R., and T. Vukicevic, 2011: Data assimilation of cloud affected radiance in a cloud resolving model. Mon. Wea. Rev., 139, 755–773, https://doi.org/10.1175/2010MWR3360.1.
Polkinghorne, R., T. Vukicevic, and F. Evans, 2010: Validation of cloud resolving model background data for data assimilation. Mon. Wea. Rev., 138, 781–795, https://doi.org/10.1175/2009MWR3012.1.
Qin, Z., X. Zou, and F. Weng, 2013: Evaluating added benefits of assimilating GOES imager radiance data in GSI for coastal QPFs. Mon. Wea. Rev., 141, 75–92, https://doi.org/10.1175/MWR-D-12-00079.1.
Roebber, P. J., 2009: Visualizing multiple measures of forecast quality. Wea. Forecasting, 24, 601–608, https://doi.org/10.1175/2008WAF2222159.1.
Schmit, T. J., M. M. Gunshor, W. P. Menzel, J. J. Gurka, J. Li, and A. S. Bachmeier, 2005: Introducing the next-generation Advanced Baseline Imager on GOES-R. Bull. Amer. Meteor. Soc., 86, 1079–1096, https://doi.org/10.1175/BAMS-86-8-1079.
Skamarock, W. C., and Coauthors, 2008: A description of the Advanced Research WRF version 3. NCAR Tech. Note NCAR/TN-475ΦSTR, 113 pp., https://doi.org/10.5065/D68S4MVH.
Skinner, P. S., L. J. Wicker, D. M. Wheatley, and K. H. Knopfmeier, 2016: Application of two spatial verification methods to ensemble forecasts of low-level rotation. Wea. Forecasting, 31, 713–735, https://doi.org/10.1175/WAF-D-15-0129.1.
Skinner, P. S., and Coauthors, 2018: Object-based verification of a prototype Warn-on-Forecast system. Wea. Forecasting, 33, 1225–1250, https://doi.org/10.1175/WAF-D-18-0020.1.
Smith, T. M., and Coauthors, 2016: Multi-Radar Multi-Sensor (MRMS) severe weather and aviation products: Initial operating capabilities. Bull. Amer. Meteor. Soc., 97, 1617–1630, https://doi.org/10.1175/BAMS-D-14-00173.1.
Stengel, M., P. Unden, M. Lindskog, P. Dahlgren, N. Gustafsson, and R. Bennartz, 2009: Assimilation of SEVERI infrared radiances with HIRLAM 4D-Var. Quart. J. Roy. Meteor. Soc., 135, 2100–2109, https://doi.org/10.1002/qj.501.
Stensrud, D. J., J.-W. Bao, and T. T. Warner, 2000: Using initial condition and model physics perturbations in short-range ensemble simulations of mesoscale convective systems. Mon. Wea. Rev., 128, 2077–2107, https://doi.org/10.1175/1520-0493(2000)128<2077:UICAMP>2.0.CO;2.
Stensrud, D. J., and Coauthors, 2009: Convective-scale Warn-on-Forecast system: A vision for 2020. Bull. Amer. Meteor. Soc., 90, 1487–1500, https://doi.org/10.1175/2009BAMS2795.1.
Stensrud, D. J., and Coauthors, 2013: Progress and challenges with Warn-on-Forecast. Atmos. Res., 123, 2–16, https://doi.org/10.1016/j.atmosres.2012.04.004.
Szyndel, M. D. E., J.-N. Thepaut, and G. Kelly, 2005: Evaluation of potential benefit of assimilation of SEVIRI water vapour radiance data from Meteosat-8 into global numerical weather prediction analyses. Atmos. Sci. Lett., 6, 105–111, https://doi.org/10.1002/asl.98.
Thompson, G., R. M. Rasmussen, and K. Manning, 2004: Explicit forecasts of winter precipitation using an improved bulk microphysics scheme. Part I: Description and sensitivity analysis. Mon. Wea. Rev., 132, 519–542, https://doi.org/10.1175/1520-0493(2004)132<0519:EFOWPU>2.0.CO;2.
Thompson, G., P. R. Field, R. M. Rasmussen, and W. R. Hall, 2008: Explicit forecasts of winter precipitation using an improved bulk microphysics scheme. Part II: Implementation of a new snow parameterization. Mon. Wea. Rev., 136, 5095–5115, https://doi.org/10.1175/2008MWR2387.1.
Thompson, G., M. Tewari, K. Ikeda, S. Tessendorf, C. Weeks, J. Otkin, and F. Kong, 2016: Explicitly-coupled cloud physics and radiation parameterizations and subsequent evaluation in WRF high-resolution convective forecasts. Atmos. Res., 168, 92–104, https://doi.org/10.1016/j.atmosres.2015.09.005.
Vukicevic, T., M. Sengupta, A. S. Jones, and T. Vonder Haar, 2006: Cloud-resolving satellite data assimilation: Information content of IR window observations and uncertainties in estimation. J. Atmos. Sci., 63, 901–919, https://doi.org/10.1175/JAS3639.1.
Wang, C., and X. Huang, 2014: Parallax correction in the analysis of multiple satellite data sets. IEEE Geosci. Remote Sens. Lett., 11, 965–969, https://doi.org/10.1109/LGRS.2013.2283573.
Wang, Y., and X. Wang, 2017: Direct assimilation of radar reflectivity without tangent linear and adjoint of the nonlinear observation operator in GSI-based EnVar system: Methodology and experiment with the 8 May 2003 Oklahoma City tornadic supercell. Mon. Wea. Rev., 145, 1447–1471, https://doi.org/10.1175/MWR-D-16-0231.1.
Weng, F., 2007: Advances in radiative transfer modeling in support of satellite data assimilation. J. Atmos. Sci., 64, 3799–3807, https://doi.org/10.1175/2007JAS2112.1.
Wheatley, D. M., K. H. Knopfmeier, T. A. Jones, and G. J. Creager, 2015: Storm-scale data assimilation and ensemble forecasting with the NSSL Experimental Warn-on-Forecast System. Part I: Radar data experiments. Wea. Forecasting, 30, 1795–1817, https://doi.org/10.1175/WAF-D-15-0043.1.
Whitaker, J. S., T. M. Hamill, X. Wei, Y. Song, and Z. Toth, 2008: Ensemble data assimilation with the NCEP Global Forecast System. Mon. Wea. Rev., 136, 463–482, https://doi.org/10.1175/2007MWR2018.1.
Yussouf, N. and K. H. Knopfmeier, 2019: Application of the Warn-on-Forecast system for flash-flood producing heavy convective rainfall events. Quart. J. Roy. Meteor. Soc., 145, 2385–2403, https://doi.org/10.1002/qj.3568.
Yussouf, N., E. R. Mansell, L. J. Wicker, D. M. Wheatley, and D. J. Stensrud, 2013: The ensemble Kalman filter analyses and forecasts of the 8 May 2003 Oklahoma City tornadic supercell storm using single- and double-moment microphysics schemes. Mon. Wea. Rev., 141, 3388–3412, https://doi.org/10.1175/MWR-D-12-00237.1.