1. Introduction
A potential candidate to replace the current aging operational Weather Surveillance Radar-1988 Doppler (WSR-88D) network in the United States is the phased array radar (PAR; Weber et al. 2007; Zrnić et al. 2007; Heinselman and Torres 2011; Weber et al. 2017). A primary advantage of the PAR is the ability to scan a volume of the atmosphere every ~1 min as opposed to every ~5 min with the WSR-88D. In an operational warning setting, this advantage of more frequent radar data usually results in severe thunderstorm and tornado warnings issued earlier with longer lead times (Bowden et al. 2015; Kuster et al. 2015; Bowden and Heinselman 2016; Wilson et al. 2017). Increasing lead times is also a goal of the National Oceanic and Atmospheric Administration (NOAA) National Severe Storms Laboratory’s (NSSL) Warn-on-Forecast (WoF) program (Stensrud et al. 2009, 2013). Thus, another promising way for frequent PAR data to potentially contribute to increases in forecast warning lead times of severe weather threats is through frequent assimilation into a storm-scale numerical weather prediction model (Yussouf and Stensrud 2010; Supinie et al. 2017).
The NSSL is developing and testing an experimental WoF system (WoFS) to provide a continuous flow of probabilistic model guidance between the National Weather Service (NWS) watch and warning temporal and spatial scales for hazardous weather threats (e.g., tornadoes, large hail, damaging wind, and flash flooding). The experimental prototype WoFS is a regional, frequently cycled, storm-scale, ensemble data assimilation (DA) and prediction system, which has demonstrated the ability to provide accurate short-term probabilistic guidance of severe thunderstorm (Wheatley et al. 2015; Yussouf et al. 2015; Jones et al. 2016; Jones et al. 2018; Skinner et al. 2018) and flash flood producing heavy rainfall events (Yussouf et al. 2016; Lawson et al. 2018; Yussouf and Knopfmeier 2019). The current experimental WoFS uses the ensemble square root Kalman filter (EnSRF; Evensen 1994; Whitaker and Hamill 2002) DA technique to assimilate WSR-88D reflectivity and radial velocity along with other available observations every 15 min at 3-km horizontal grid spacing. The relatively coarse assimilation frequency and grid spacing is employed due to the rigorous computational requirements of a real-time system. Numerous past studies have demonstrated the potential benefits of more frequent DA cycling (Xue et al. 2006; Yussouf and Stensrud 2010; Schenkman et al. 2011; Sobash and Stensrud 2015) and finer horizontal grid spacings (Potvin and Flora 2015; Sobash et al. 2019). Therefore, with the exponential increase in computational resources, a future WoFS will likely implement more frequent DA at a finer horizontal grid spacing.
The potential benefits of assimilating radar data more frequently have been explored by previous studies using observing system simulation experiments (OSSEs; Zhang et al. 2004; Xue et al. 2006; Lei et al. 2007; Yussouf and Stensrud 2010; S. Wang et al. 2013). Zhang et al. (2004) compared 2- and 5-min DA cycling intervals and found that the more frequent DA cycling only slightly improved the first few analyses before the differences between the two cycling intervals became negligible. Xue et al. (2006) found that assimilating radar volumetric data every 1 min rather than every 2.5 and 5 min generally resulted in better analyses of storms for at least the first few DA cycles. With a specific focus on PAR, Lei et al. (2007) used OSSEs to conclude that assimilating radar volumetric data every 1.25 min can reduce analysis errors faster than assimilating data every 5 min. They also note that this result is particularly useful in situations when newly developed storms need to be quickly assimilated into the analyses. Yussouf and Stensrud (2010) used more sophisticated synthetic radar observations in their OSSEs by emulating operational scanning strategies for the WSR-88D and PAR. Their results agree with Lei et al. (2007) by showing that the more frequent PAR observations lead to more accurate storm-scale analyses and short-term forecasts of convective weather after 15 min of data assimilation with 1-min cycling intervals. They also found that the differences between their WSR-88D and PAR experiments are minimal after 60 min of data assimilation. Generally, these OSSE studies found that more frequent radar DA cycling can benefit storm-scale analyses and short-term forecasts of severe convective weather.
Even though OSSEs can provide useful conclusions and guidance for future research, their results are generally too optimistic due to not accounting for other sources of error, such as model and real observation errors. Rigorous testing of the impact of PAR in storm-scale modeling is necessary to assess the next-generation PAR technology beyond the current WSR-88D network for the WoFS. Also, as mentioned earlier, one attractive feature of PAR technology is the flexible high temporal frequency volume scan capability. Therefore, this study’s goal is to assimilate real PAR volumetric data and use full model physics to determine the optimal temporal frequency of PAR observations for a WoFS-type ensemble storm-scale DA and forecast system. Based on the previous OSSE studies, we approach this problem by first exploring experiments using an equivalent EnSRF to synchronously assimilate PAR volumetric data with various cycling intervals (e.g., 5-min PAR volumetric data are assimilated using 5-min cycling intervals). Furthermore, computational constraints for running the experimental WoFS in real-time need to be considered, so an additional experiment is performed to explore the concept of adaptive cycling intervals. Essentially, the cycling interval is adjusted after several cycles from more frequent DA cycling to less frequent DA cycling.
The previously mentioned studies assimilated radar data using synchronous EnSRF DA methods, so only the nearest volume or elevation scans in time were used at the time of assimilation. However, the frequent stopping and restarting of a model to assimilate temporally dense radar observations can introduce imbalances that are avoided with larger cycling intervals (Lange and Craig 2014 and references therein). Even so, while computationally cost saving, the use of longer cycling intervals can introduce observation timing errors or miss details in a storm’s evolution (S. Wang et al. 2013) if observations further from the time of DA are discarded. S. Wang et al. (2013) introduced the 4D ensemble square root filter (4DEnSRF) to take advantage of both the longer DA cycling intervals and the ability to assimilate data at asynchronous observation times. Using radar-based OSSEs, S. Wang et al. (2013) concluded the 4DEnSRF produces more accurate analyses and forecasts than the 3D EnSRF for cycling intervals > 1 min while being computationally more cost efficient. Supinie et al. (2017) conducted storm-scale DA and forecast experiments using real observations from a WSR-88D radar and NSSL’s National Weather Radar Testbed (NWRT) PAR (Forsyth et al. 2005) along with full model physics. They used a 5-min cycling interval with the 4DEnSRF, so up to five full volumes of PAR data were asynchronously assimilated every 5 min while only a single volume of WSR-88D data were assimilated at each DA time. Results from their experiments show the more frequent PAR volumetric data lead to more accurate analyses and forecasts, especially for shorter assimilation periods (i.e., <30 min). The relative beneficial impact of the PAR data does decrease with longer assimilation periods (i.e., >45 min), which is consistent with the previously mentioned EnSRF OSSE studies. Similar to Supinie et al. (2017), we investigate an experiment using 4DEnSRF to asynchronously assimilate 1-min PAR volumetric data with a 5-min cycling interval.
Storm-scale analyses and forecasts from the synchronous and asynchronous DA cycling experiments are evaluated and compared using subjective assessments and objective verification techniques, including neighborhood and object-based methods. For this study, we assimilate NWRT PAR observations from the 31 May 2013 El Reno, Oklahoma, tornado event onto a grid with 1-km horizontal grid spacing and use full model physics. The next section provides a brief summary of the 31 May 2013 tornado event. Details about the PAR observations, DA and forecast systems, experiment design, and evaluation and verification methods are specified in section 3. Results from the various experiments are presented in section 4. Finally, section 5 provides a summary and discussion of the results.
2. Overview of the 31 May 2013 tornado event
Shortly after 2130 UTC 31 May 2013, storms began developing west of El Reno in an environment supportive of tornadic supercells (NOAA/NWS 2013; Bluestein et al. 2015). The NWS Weather Forecast Office (WFO) in Norman, Oklahoma, issued the first severe thunderstorm warning for these initial storms at 2146 UTC. At 2236 UTC, NWS Norman issued the first tornado warning for the supercell west of El Reno. The primary tornado of interest, termed the “El Reno tornado” for this study, began at 2303 UTC west-southwest of El Reno, skirted the southern edge of El Reno in mostly open fields, and dissipated east of El Reno near I-40 around 2344 UTC (Fig. 1b; see Wakimoto et al. 2016 for a more detailed analysis). Unfortunately, the El Reno tornado was responsible for 8 fatalities and 26 injuries (NOAA/NWS 2013). The NWS rated this tornado an EF3 with a pathlength of approximately 26 km and a maximum path width of about 4.2 km. Two mobile radars scanning the tornado measured near-surface winds > 100 m s−1, which highlights the strength of the tornado and the accompanying storm (Snyder and Bluestein 2014; Wurman et al. 2014). After the El Reno tornado dissipated, the supercell cycled and produced an EF1 tornado, which lasted from 2354 to 0009 UTC, in western parts of Oklahoma City, Oklahoma (Fig. 1b). Additional details about this event’s storm environment, tornadoes, and flash flooding are provided by NOAA/NWS (2013), NOAA (2014), Wurman et al. (2014), Snyder and Bluestein (2014), Bluestein et al. (2015), Wakimoto et al. (2015), Wakimoto et al. (2016), and Yussouf et al. (2016).
(a) The multiscale model domains at 15-, 3- and 1-km horizontal grid spacing and (b) the 1-km grid-spacing storm-scale domain. In (b), damage swaths of the EF3 tornado near El Reno and subsequent EF1 tornado near Oklahoma City are shaded in orange; light and dark gray dashed lines represent the edges of the PAR scanning sectors for 2145–2215 UTC and 2221–2300 UTC, respectively; and light blue dashed lines outline the area where eFSS is computed. (c) Multiscale and (d) storm-scale data assimilation and forecast timelines.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
3. Data, models, and methods
a. NWRT PAR observations
The NWRT PAR began scanning for storms around 1655 UTC 31 May 2013 and finished scanning around 0355 UTC 1 June 2013. During this time period, the PAR successfully interrogated the El Reno storm from convective initiation around 2130 UTC through the entire evolution of the El Reno tornado with the exception of a 5-min period from 2216 to 2221 UTC when the horizontal scanning sector was shifted ~10°. Volumetric data intervals incrementally increased from about 45 to 69 s before the sector shift owing to additional elevation angles being added to the volume scans. After the sector shift, volume scanning intervals remained nearly constant at about 71 s.
Prior to this study, the PAR reflectivity and radial velocity observations were manually quality controlled using the NCAR Earth Observing Laboratory’s Solo-II software for radar volumes between 2140 and 0000 UTC. Areas where reflectivity is less than 20 dBZ are set to 0 dBZ for reflectivity and missing (i.e., no observations are assimilated) for radial velocity. For DA preparation, the quality-controlled PAR observations are bilinearly interpolated onto the 1-km horizontal grid domain while vertically remaining on the original tilts (see, e.g., Xue et al. 2006; Supinie et al. 2017). The original PAR scans consisted of 19 elevation angles ranging from 0.50° to 52.90°, but only elevation angles at and below 8° are used in this study.
To help suppress spurious convection in the model forecasts, 0 dBZ (i.e., clear-air reflectivity) is added to the gridded PAR observations where reflectivity from the Oklahoma City WSR-88D (KTLX) radar is less than or equal to 0 dBZ outside of the 90° PAR scanning sector. Areas where KTLX’s reflectivity is greater than 0 dBZ are set to missing.
b. Multiscale DA and forecast system
A multiscale DA and forecast system with 36 ensemble members is used to provide initial and lateral boundary conditions for the storm-scale DA and forecast system. The outer domain covers the CONUS with 15-km horizontal grid spacing and 340 × 235 × 51 grid points (Fig. 1a). The inner domain is centered on Oklahoma within the coarser domain and has 3-km horizontal grid spacing and 401 × 401 × 51 grid points (Fig. 1a). The first 18 members from the NCEP’s Global Ensemble Forecast System (GEFS; Toth et al. 2004; Wei et al. 2008) and North American Mesoscale Forecast System (NAM; soil only) provide the boundary conditions for the 15-km ensemble and the initial conditions for the both the 3- and 15-km ensembles. The 15-km ensemble provides the boundary conditions for the nested 3-km ensemble.
Both ensembles are run simultaneously from 0000 UTC 31 May 2013 to 0000 UTC 1 June 2013 using the Advanced Research version of the Weather Research and Forecasting (WRF-ARW, version 3.9.1.1; Skamarock et al. 2008) Model for the forecast system and the community-based Gridpoint Statistical Interpolation (GSI, version 3.4; Hu et al. 2015a) system with EnKF (version 1.0; Hu et al. 2015b) for the DA system. The different physics parameterization combinations that are used to create the ensemble diversity are the same as in Table 2 of Yussouf et al. (2015). The physics combinations are the same for both ensembles, but cumulus parameterization is not used for the 3-km ensemble. Also, the NSSL two-moment microphysics scheme (option 17 in WRF; Mansell et al. 2010) and the Noah (Tewari et al. 2004) land surface schemes are used in both ensembles. Only conventional observations (e.g., surface weather, radiosonde, and aircraft-based data) from the NCEP’s prepbufr files are assimilated hourly onto the multiscale grid domains.
c. Storm-scale DA and forecast system
The storm-scale DA and forecast system is integrated over a 401 × 401 gridpoint domain with a fine, 1-km horizontal grid spacing (compared to the current 3-km version of WoFS) and 51 vertical levels (Fig. 1b). The 3-km ensemble analyses provide the initial and lateral boundary conditions for the 1-km domain starting at 2100 UTC (Figs. 1c,d). Next, a 45-min 36-member ensemble forecast is initialized at 2100 UTC to spin up the model fields and to provide a background ensemble forecast for the first storm-scale DA at 2145 UTC (Fig. 1d). The same version of WRF-ARW and the same physics combinations as the multiscale forecast system are used for the storm-scale forecast system. However, the Thompson microphysics scheme (option 8 in WRF; Thompson et al. 2008) is used instead of the NSSL two-moment microphysics scheme due to the DA system for the storm-scale domain (see below) not being able to work with the NSSL two-moment microphysics scheme. The Thompson microphysics scheme is currently used in some operational storm-scale models, such as the High-Resolution Rapid Refresh (HRRR; Benjamin et al. 2016) model. Also, no storm is advected into the domain through the lateral boundaries and, therefore, the impact of using a different microphysics scheme in the storm-scale domain (i.e., Thompson microphysics scheme) would be minor.
Starting from the background ensemble forecast at 2145 UTC, DA experiments are performed for 75 min until 2300 UTC (Fig. 1d). The processed PAR reflectivity and radial velocity observations are assimilated during this time period using the 4DEnSRF algorithm (S. Wang et al. 2013) in the Advanced Regional Prediction System’s (ARPS; Xue et al. 2003) EnKF DA system (Y. Wang et al. 2013). The 4DEnSRF algorithm is used for all experiments for a fairer comparison between the synchronous and asynchronous experiments (S. Wang et al. 2013). When observations are assimilated synchronously, the 4DEnSRF implementation is equivalent to the parallel EnSRF (Anderson and Collins 2007). Radial velocity is only assimilated in areas where reflectivity is greater than 10 dBZ. The standard deviations of the PAR reflectivity and radial velocity observations errors are assumed to be 7 dBZ and 3 m s−1, respectively. Spatial covariance localization is based on the fifth-order correlation function from Gaspari and Cohn (1999) and uses a radius of influence of 6 km in both the horizontal and vertical directions. For experiments assimilating asynchronous data, the temporal covariance localization uses a radius of influence equal to half the cycling interval (e.g., 2.5-min temporal radius of influence for 5-min cycling interval). Reflectivity DA is used to update perturbation potential temperature, the vertical wind component, and microphysics variables (i.e., water vapor, cloud, rain, snow, graupel, and ice mixing ratios and rain and ice number concentrations), while radial velocity DA is used to update the three wind components.
Two covariance inflation techniques are used to help maintain ensemble spread during DA cycling. A 20% multiplicative inflation factor (Anderson 2001) is applied to regions in the prior ensemble where reflectivity is greater than 5 dBZ. After assimilating data, relaxation-to-prior spread (RTPS; Whitaker and Hamill 2012) with a relaxation factor of 0.98 is applied to all model state variables across the entire domain. Finally, ensemble forecasts are launched from analyses every 15 min from 2200 to 2300 UTC and are run until 0000 UTC 1 June 2013 (Fig. 1d).
d. Experiment design
For this study, three sets of experiments are conducted to help determine the optimal temporal frequency of PAR data. The first set of experiments is designed to emulate the OSSEs that explored the impact of radar DA cycling intervals on storm-scale analyses and forecasts (e.g., Zhang et al. 2004; Xue et al. 2006). These experiments synchronously assimilate PAR data every 1, 3, 5, and 15 min and are named PAR1Cyc1, PAR3Cyc3, PAR5Cyc5, and PAR15Cyc15, respectively (Fig. 2). Ensemble forecasts are initialized from each of those experiments every 15-min with WRF history files output every 5 min through 0000 UTC the next day (Fig. 1d). Another synchronous DA experiment is conducted to demonstrate the potential role adaptive cycling intervals may play in a future WoFS using PAR observations. During the early stages of storm development (i.e., 2145–2200 UTC), the 1-min DA cycling interval is used to spin up the storm in the model before switching to the 15-min DA cycling interval for the remaining time (i.e., 2200–2300 UTC). This experiment is named Cyc1+Cyc15 (Fig. 2) and will be compared to the PAR15Cyc15 experiment, which is the current WoFS DA cycling frequency (Skinner et al. 2018) for real-time demonstration.
Schematic of the data assimilation experiments. Red and blue vertical lines indicate the times PAR volumetric data are assimilated. Longer and shorter vertical lines in PAR1Cyc5 represent the centers of the data assimilation windows and PAR volumetric data, respectively.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
The final set of experiments compare the impact of asynchronously assimilating 1-min PAR volumetric data using 4DEnSRF with a 5-min cycling interval (PAR1Cyc5) to the previous PAR5Cyc5 experiment (Fig. 2). For this asynchronous experiment, the assimilation window is equal to the cycling interval, so all observations between −2.5 and +2.5 min are assimilated for each DA cycle.
e. Evaluation and verification methods
A combination of subjective evaluations and objective verification techniques are used to assess and compare the ensemble analyses and forecasts from the various experiments. To assess ensemble filter performance within the DA period, mean innovation, root-mean-square innovation (RMSI), total ensemble spread, and consistency ratio diagnostics are computed in observation space for reflectivity and radial velocity where observed or model reflectivity is greater than 15 dBZ. Mean innovation, total ensemble spread, and consistency ratio are computed using the equations in Dowell and Wicker [2009; Eqs. (3.1), (3.3), (3.4)], but RMSI, which is also in the denominator of the consistency ratio, is computed following Dowell et al. [2011; Eq. (4.1)]. Mean innovation is a measure of the model bias, so positive (negative) values indicate the model underforecasts (overforecasts) the intensity and/or areal coverage of reflectivity and radial velocity. RMSI is a measure of how much the model fits the observations, so smaller values indicate the model is closer to the observations. Total ensemble spread is a measure of ensemble spread in conjunction with the observation error, so higher values indicate more spread, while the lowest attainable total ensemble spread is the observation error. Consistency ratio is a measure of the balance between total ensemble spread and RMSI, so an ensemble is considered to be overdispersive (underdispersive) when consistencies are greater than (less than) 1. Consistency values near 1 are optimal. These diagnostics are computed for the background forecasts prior to multiplicative inflation and analyses prior to the application of RTPS.
Two diagnostic fields are used to assess the ensemble forecasts. First, ensemble probability plots of 5-, 30-, and 60-min forecasts of reflectivity at 2 km above mean sea level (MSL) are subjectively compared to a gridded mosaic of observed 2 km MSL reflectivity. Reflectivity observations from multiple WSR-88D radars [i.e., KTLX, Frederick, Oklahoma (KFDR), Vance Air Force Base, Oklahoma (KVNX), and Tulsa, Oklahoma (KINX)] are merged (Lakshmanan et al. 2006) onto a grid domain with 1-km horizontal grid spacing using the Warning Decision Support System–Integrated Information (WDSS-II; Lakshmanan et al. 2007) program suite within the Multi-Radar Multi-Sensor (MRMS; Smith et al. 2016) system. The gridded reflectivity is then interpolated onto the 1-km model domain in Fig. 1b for comparison with the model forecasts.
For objective verification of reflectivity, the ensemble fractions skill score (eFSS; Duc et al. 2013) is computed for 2 km MSL reflectivity greater than 35 dBZ using neighborhood widths of 0, 2, 4, 8, 16, 32, 64, 128, and 256 km for the 160 km × 120 km area illustrated in Fig. 1b. The eFSS is similar to the traditional FSS (Roberts and Lean 2008), but in addition to the spatial neighborhood probabilities, eFSS uses neighborhood probabilities extended into the ensemble space (Duc et al. 2013). For each experiment, average eFSS values are computed for each neighborhood width using forecast history files every 5 min during the first hour after initialization. An eFSS value of 1 indicates a forecast with perfect neighborhood probabilities (i.e., no frequency bias). To provide further insight, a reference FSS, FSSref, is computed using the observed frequency fo to determine the halfway point between a random forecast and a perfect forecast (i.e., FSSref = 0.5 + fo/2; same as FSSuniform in Roberts and Lean 2008). The neighborhood widths at which eFSS = FSSref are determined for forecasts every 5 min within the first hour after initialization starting with the 5-min forecast. The best possible neighborhood width at which eFSS = FSSref is 0 km, which means eFSS ≥ FSSref when the neighborhood size is one grid point.
Since the El Reno storm had a strong rotating updraft, the other diagnostic variable used to assess the ensemble forecasts is 2–5-km updraft helicity (UH; Kain et al. 2008), which serves as a proxy to rotating updrafts in models. Forecasts of ensemble probabilities of instantaneous 2–5-km UH greater than 400 m2 s−2 are aggregated together using output from every 5 min during the entire forecast period. This UH threshold is based on the subjectively determined threshold used in defining objects, as described in the following paragraphs. Following Skinner et al. (2016), the UH field for each ensemble member is smoothed before computing the ensemble probabilities by first finding the maximum UH within a 3 × 3 gridpoint neighborhood centered on each grid point. Next, a Gaussian kernel with a standard deviation of 2 grid points is applied using only grid points within a 5 × 5 gridpoint neighborhood centered on each grid point. In addition, forecasts of ensemble 90th percentile intensities of 2–5-km UH at each grid point are aggregated together like the ensemble probabilities to provide a means to compare mesocyclone intensities via UH among the experiments’ forecasts. These UH probabilities and intensities are subjectively evaluated using the maximum azimuthal wind shear (Smith and Elmore 2004; Miller et al. 2013) in the 2–5-km layer as a proxy for midlevel rotation in mesocyclones.
An object-based verification technique is used to quantitatively assess the performance of the forecasts of the El Reno storm’s midlevel mesocyclones. This technique is similar to the one presented in Skinner et al. (2018). However, several differences exist between the two methods, so our process of determining, merging, and matching objects is thoroughly detailed for comparison. Also, Skinner et al. (2018) used percentiles to determine their thresholds, but since the same model configurations are used for all experiments, our thresholds are subjectively determined through visually ascertaining what does and does not constitute an object in individual ensemble member forecast and verification fields. Even so, the percentiles for the arbitrarily determined thresholds ended up being similar for both the UH and azimuthal wind shear fields.
First, instantaneous 2–5-km UH forecast and azimuthal wind shear verification fields are thresholded by setting values less than 400 m2 s−2 and 0.008 s−1, respectively, to zero. Next, the thresholded fields are smoothed using a Gaussian filter with a kernel standard deviation of 1. The values in the smoothed fields are then normalized back to the values in the original fields using the ratio between the maximum value in the smoothed fields and the maximum value in the original fields. In Skinner et al. (2018), objects were merged using a minimum spatial displacement, but for our study, the smoothing of the fields is used to merge objects. Next, the normalized, smoothed fields are used to create binary fields, where values less than 200 m2 s−2 and 0.005 s−1, respectively, are set to zero and values greater than or equal to those thresholds are set to one. Distinct objects in the binary fields are then assigned numerical labels (e.g., 1, 2, …). These object labels are then applied to the original unsmoothed, thresholded fields. Finally, the original thresholded fields, along with the object labels, are used to compute various object attributes, such as centroid location, area, maximum intensity, and eccentricity, for each object. This object identification process is similar to the Method for Object-based Diagnostic Evaluation (MODE; Davis et al. 2006a,b) methodology. Forecast and verification objects with maximum intensities greater than 700 m2 s−2 and 0.012 s−1, respectively, areas greater than 25 km2, and eccentricities less than 1 are used in the object matching and verification. Unlike Skinner et al. (2018), no temporal continuity threshold is applied when selecting objects for verification.
For objects to be considered a match, the centroids of forecast objects must synchronously exist within 40 km of the centroids of verification objects. Skinner et al. (2018) computed a total interest score (Davis et al. 2006a), which accounts for centroid and minimum spatial displacements and temporal displacements, for all possible object matches. However, for this study, only the centroid displacement is used to match objects. Thus, the forecast object with the smallest centroid displacement error is considered a match. If there are additional forecast objects within the 40-km range, they are considered to be unmatched. If no forecast objects exist within 40 km, the verification object goes unmatched. This object-matching information is then used to form a contingency table with hits a, false alarms b, and misses c. Since no more than one verification object exists at a time, the maximum value for hits and misses is 1. If more than one forecast object exists within the 40-km range, the extra forecast objects get added to the false alarm category. A verification object exists for the El Reno supercell every 5 min from 2300 to 0000 UTC, except at 2345 UTC; therefore, no contingency table components are included at that time. The contingency table components from individual forecasts are aggregated together for each experiment’s entire ensemble of forecasts. With these aggregated contingency table components, probability of detection [POD = a/(a + c)], false alarm ratio [FAR = b/(a + b)], frequency bias [bias = (a + b)/(a + c)], and critical success index [CSI = a/(a + b + c)] are computed and presented using performance diagrams (Roebber 2009). The best possible forecast results in POD = 1, FAR = 0, bias = 0, and CSI = 1, so better forecasts end up closer to the top-right corner of the performance diagram.
4. Results
a. Synchronous DA cycling interval experiments
1) Observation-space diagnostics
Innovation diagnostics for the background forecasts and analyses during the 75-min DA period indicate large differences among the synchronous cycling interval experiments. All experiments’ background forecasts of reflectivity begin with a mean innovation of about −24 dBZ, indicating reflectivity is overpredicted in the spinup forecasts, and a forecast RMSI of about 27 dBZ (Figs. 3a,b). PAR1Cyc1’s background reflectivity forecasts have mean innovations closest to 0 dBZ for most of the DA cycling period and the smallest RMSIs for the entire DA cycling period (Figs. 3a,b). However, mean innovations and RMSIs are larger for experiments with longer DA cycling intervals due to having more time for errors to grow during the forecast periods in between DA times. Some of the error growth is likely due to the Thompson microphysics scheme having a high bias in reflectivity (Skinner et al. 2018). This high bias is represented by larger negative mean innovations for experiments with longer DA cycling intervals (Fig. 3a).
Synchronous DA experiments’ observation-space diagnostics, including mean innovation, RMSI, total ensemble spread, and consistency ratio, for background and analysis reflectivity and radial velocity for synchronous DA experiments.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
Similar to RMSI, the total ensemble spread is larger for experiments with longer DA cycling intervals (Fig. 3c). PAR1Cyc1’s total ensemble spread is nearly equal to the observation error standard deviation of 7 dBZ at the end of DA cycling, indicating little spread in the ensemble forecasts (Fig. 3c). However, PAR1Cyc1’s consistency ratios are generally around one during the entire DA cycling period, which indicates the ensemble has good spread relative to RMSI (Fig. 3d). PAR3Cyc3 and PAR5Cyc5 also exhibit good ensemble spread with consistency ratios generally between 0.8 and 1.0 (Fig. 3d). Conversely, PAR15Cyc15’s consistency ratio is never larger than 0.7, which indicates the ensemble is underdispersed during the entire DA cycling period (Fig. 3d). Similar observation-space diagnostics results are found for radial velocity (Figs. 3e–h). Also, there are no signs of filter divergence for the cycling interval experiments. Overall, shorter cycling intervals yield background forecasts with smaller mean innovations, RMSIs, and total ensemble spread. Shorter cycling intervals also result in faster decreases in mean innovation, RMSI, and total ensemble spread while maintaining better consistency ratios.
2) Reflectivity analyses and forecasts
After only 15 min of DA cycling, ensemble probability plots of the analyses (5-min forecasts valid at 2205 UTC are used here as a proxy for the analyses at 2200 UTC) of reflectivity at 2 km MSL reveal differences already exist across the spectrum of cycling interval experiments (Figs. 4b–e). Specifically, more spurious convection exists in PAR15Cyc15’s ensemble analyses to the south and to the east of the actual storms, as indicated by higher probabilities of reflectivity (Fig. 4e). This issue is the least problematic for PAR1Cyc1, which suppresses most of the spurious convection (Fig. 4b). PAR3Cyc3 and PAR5Cyc5 produce ensemble analyses that fall in the middle of the spurious convection spectrum (Figs. 4c,d). For 30-min forecasts, the coverage and amount of spurious convection is larger for experiments with longer cycling intervals (Figs. 4g–j). PAR1Cyc1’s 30-min reflectivity forecast is most similar to the observed reflectivity due to better capturing the southern-end storm and having less spurious convection to the southwest and northeast of the main storm (Fig. 4g). At 60 min, the differences among the cycling interval experiments are similar to those at the earlier forecast times (Figs. 4l–o). All experiments depict a strong storm at about the same latitude as the actual storm, but PAR1Cyc1 is more focused on the main storm with less spurious convection. As the 30- and 60-min forecasts show, all experiments suffer from forecast storms propagating too fast to the east (Figs. 4g–j, l–o). This propagation bias in model forecasts is a known issue numerous storm-scale modeling studies (e.g., Vandenberg et al. 2014; Yussouf et al. 2015; Supinie et al. 2017) have previously noted and is beyond the scope of this study.
Observed 2 km MSL reflectivity at (a) 2205, (b) 2230, and (c) 2300 UTC, and ensemble probabilities of 2 km MSL reflectivity > 35 dBZ for (b)–(e) 5-, (g)–(j) 30-, and (l)–(o) 60-min forecasts, which are initialized from the synchronous DA experiments’ 2200 UTC analyses, valid at 2205, 2230, and 2300 UTC 31 May 2013, respectively. The observed 35-dBZ reflectivity contour is overlaid in black on the forecasts. The black dot is the location of NWRT PAR.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
After 75 min of DA cycling, ensemble probability plots of the reflectivity analyses from the cycling interval experiments reveal similar results to those after 15 min of DA cycling; longer cycling intervals have more spurious convection (Figs. 5b–e). All experiments maintain a strong storm through the rest of the forecast period (not shown), but the longer cycling interval experiments, especially PAR15Cyc15, also maintain the spurious convection, which negatively impacts the forecast (Figs. 5g–j, l–o). By 0000 UTC, PAR1Cyc1, PAR3Cyc3, and PAR5Cyc5 have a better handle on developing new convection to the west of the El Reno storm (Figs. 5k–o). While not the focus of this paper, a deadly flash flooding event in the Oklahoma City metro occurred due to new convection continuously developing to the west of the El Reno storm (Yussouf et al. 2016), so capturing this trailing convection in the forecasts is important.
As in Fig. 4, but for ensemble forecasts initialized at 2300 UTC and valid at 2305, 2330, and 0000 UTC.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
At the end of the forecast period, PAR1Cyc1, PAR3Cyc3, and PAR5Cyc5 all have similar locations of the El Reno storm, while PAR15Cyc15’s storm is too far southeast (Figs. 5k–o). Except for maybe PAR5Cyc5, all experiments have cold pools near El Reno that are colder than what the El Reno mesonet station observed at 2300 UTC (Fig. 6). Interestingly, PAR15Cyc15’s cold pool is the coldest and largest in areal coverage (Fig. 6) likely owing to a combination of having larger increments within areas of observed reflectivity, as discussed by Dowell et al. (2011), and more spurious convection. In fact, several of PAR15Cyc15’s ensemble members have a southeastward surging cold pool that intensifies the spurious convection located to the south of the El Reno storm (not shown). Other PAR15Cyc15 members maintain the original El Reno storm (not shown), but the stronger cold pool results in the storm propagating too far to the southeast. A combination of both scenarios results in the higher probabilities of reflectivity extending too far south (Fig. 5o).
Ensemble probability-matched mean (Ebert 2001) temperature (°C; colored shading) along with ensemble mean rain mixing ratio (dark gray contours at 0.1, 2, and 5 g kg−1) and wind data [kt (1 kt ≈ 0.51 m s−1); short gray wind barbs] at the lowest model level (~9 m AGL) from the synchronous experiments’ 2300 UTC analyses. The 9-m temperature (°C; colored circles) and 10-m wind data (kt; long black wind barbs) from select Oklahoma Mesonet stations at 2300 UTC are also shown. For reference, the El Reno mesonet station is marked with a star. The average minimum temperatures at the lowest model level for the area under the El Reno storm are annotated in the bottom-left corner of each panel.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
For all experiments, forecast reflectivity is on average too low in the low levels during the DA cycling, so positive rain and graupel mixing ratios and rain number concentration increments are repeatedly added to the low levels (not shown). Temperature is only updated through the ensemble covariances, so because reflectivity and temperature are likely negatively correlated, increases in reflectivity likely result in decreases in temperature in the low levels, as Dowell et al. (2011) also discussed. Therefore, for shorter cycling intervals, such as for PAR1Cyc1 and PAR3Cyc3, cold pools likely become too cold due to more frequent reflectivity observations being assimilated and updating the microphysics variables.
Using neighborhood-based verification, PAR1Cyc1 has the largest average eFSS values for neighborhood widths larger than ~16 km (Fig. 7a). In fact, PAR1Cyc1’s eFSS asymptotes to the largest eFSS value for the largest neighborhood widths as a result of having the smallest frequency bias within the subdomain, likely due to having less spurious convection (Fig. 7a). Conversely, PAR15Cyc15 has the smallest eFSS values for all neighborhood sizes at least partially owing to having the most spurious convection (Fig. 7a). For neighborhood widths smaller than 16 km, PAR1Cyc1, PAR3Cyc3, and PAR5Cyc5 have similar eFSS values (Fig. 7a).
(a) Average eFSS for each neighborhood width (km) using all five 1-h forecasts of 2 km MSL reflectivity > 35 dBZ and (b) neighborhood widths (km) where eFSS = FSSref for each forecast output time. Short black lines on the x axis in (b) demarcate the forecast initialization times.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
While all of the experiments’ forecasts improve with later initialization times, shorter cycling interval experiments generally achieve FSSref at smaller neighborhood widths than longer cycling interval experiments (Fig. 7b). In fact, during the first hour after all initializations, PAR1Cyc1’s forecasts outperform the other experiments’ forecasts for most forecast output times (Fig. 7b). For forecasts initialized at 2300 UTC, PAR3Cyc3 and PAR5Cyc5 achieve FSSref at smaller neighborhood widths than PAR1Cyc1 starting around 2345 UTC (Fig. 7b). However, this result is due to a combination of PAR3Cyc3 and PAR5Cyc5 forecasting more spurious convection, as indicated by the smaller eFSS values at the larger neighborhood widths (Fig. 7a), and PAR1Cyc1 not forecasting enough new convection to the west of the El Reno storm (Fig. 5n). Overall, the results from the reflectivity forecasts indicate more frequent PAR DA can improve forecasts by more quickly developing convection in the correct locations while removing spurious convection.
3) 2–5-km UH forecasts
All experiments, except for PAR15Cyc15, forecast greater than 50% probabilities of 2–5-km UH > 400 m2 s−2 over the areas of the observed mesocyclones (e.g., azimuthal wind shear > 0.012 s−1) responsible for the El Reno and Oklahoma City tornadoes with only 15 min of DA (Figs. 8a–d). More specifically, PAR1Cyc1 is the only experiment that has probabilities greater than 95% for a portion of the El Reno storm’s observed azimuthal wind shear track (Fig. 8a). Both the probabilities (Fig. 8a) and UH magnitudes (Fig. 8e) reveal that PAR1Cyc1 forecasts less spurious rotation to the north and south of the El Reno storm with PAR15Cyc15 having the most spurious rotation. Except for the spatial differences, all of the experiments forecast similar maximum 2–5-km UH intensities (Figs. 8e–h).
0–2-h forecasts initialized at 2200 UTC of (a)–(d) probabilities of 2–5-km UH greater than 400 m2 s−2 and (e)–(h) ensemble 90th percentile intensities of 2–5-km UH (m2 s−2). Black contours are azimuthal wind shear at 0.006 s−1 and 0.012 s−1 accumulated from 2200 to 0000 UTC. The black dot is the location of the NWRT PAR.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
After 75 min of DA cycling, PAR1Cyc1, PAR3Cyc3, and PAR5Cyc5 similarly forecast swaths of high probabilities and intense maximum values of 2–5-km UH that mostly overlap the azimuthal wind shear track (Figs. 9b–d,f–h). Among those three experiments, PAR1Cyc1 has the least amount of spurious rotation to the north and south of the El Reno storm, has a more focused UH track, and extends the highest probabilities farther to the east along the azimuthal wind shear track (Figs. 9b–d,f–h). Even though PAR15Cyc15 performs well early in the forecast period, spurious UH develops to the south of the El Reno storm’s azimuthal wind shear track (Figs. 9d,h). This spurious rotation arises from a combination of spurious convection existing to the south of the El Reno storm and the forecast El Reno storm propagating to the southeast away from the azimuthal wind shear track due to the stronger cold pool.
As in Fig. 8, but for 0–1-h forecasts initialized at 2300 UTC.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
Using object matching to compute contingency table statistics, objective verification of 2–5-km UH forecasts reveals an interesting progression of forecast performance. For PAR1Cyc1, an increase in the number of 2–5-km UH objects from the forecasts initialized at 2200 UTC and 2215 UTC (Table 1) results in an increase in bias and a small decrease in CSI (Fig. 10a). However, starting with the forecast initialized at 2230 UTC, PAR1Cyc1’s subsequent forecasts result in smaller biases and increasingly better CSI values, with the final forecast yielding a CSI of ~0.8 and a bias of ~1.0 (Fig. 10a). For PAR3Cyc3 and PAR5Cyc5, this consistent improvement toward substantially smaller biases and larger CSIs occurs starting with forecasts initialized at 2245 and 2300 UTC, respectively (Figs. 10b,c). Conversely, PAR15Cyc15 never substantially improves due to only assimilating six volumes of PAR data (Fig. 10d). Except for PAR15Cyc15, this shift in forecast performance signifies the point at which most of the UH objects are associated with the El Reno storm and are not considered spurious (Table 1). The later turn toward better performance by PAR3Cyc3 and PAR5Cyc5 results in three out of the top five forecasts being produced by PAR1Cyc1 (Fig. 10). Therefore, more frequent DA cycling leads to a quicker progression from forecasts with mostly spurious rotating storms to forecasts with mostly nonspurious rotating storms.
Total number of UH objects within 40 km of the azimuthal wind shear objects for each experiment’s ensemble forecast. The optimal number of objects is 432 (i.e., 36 ensemble members × 12 azimuthal wind shear objects).
Performance diagram for 2–5-km UH objects from forecasts initialized at 1) 2200, 2) 2215, 3) 2230, 4) 2245, and 5) 2300 UTC for (a) PAR1Cyc1, (b) PAR3Cyc3, (c) PAR5Cyc5, and (d) PAR15Cyc15. PAR1Cyc1, PAR3Cyc3, and PAR5Cyc5’s 2300 UTC forecast performances are also plotted in (d) with increased transparency.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
b. Example of adaptive cycling intervals
By assimilating PAR volumetric data more frequently for the first 15 min of the DA period, Cyc1+Cyc15 is able to produce better reflectivity forecasts of the El Reno storm than PAR15Cyc15. Subjectively, Cyc1+Cyc15’s forecasts have less spurious convection at all forecast times (Fig. 11). Also, with less propagation to the southeast, Cyc1+Cyc15’s El Reno storm is latitudinally more correct than the storm in PAR15Cyc15’s forecasts (Fig. 11). Objectively, Cyc1+Cyc15’s reflectivity forecasts have substantially higher eFSS values at all scales and achieve FSSref at substantially smaller scales at most forecast times (Fig. 12).
As in Fig. 5, but for Cyc1+Cyc15 and PAR15Cyc15.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
As in Fig. 7, but for Cyc1+Cyc15 and PAR15Cyc15.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
Probability and intensity forecasts of 2–5-km UH also reveal large differences between the two experiments. Cyc1+Cyc15 has less spurious rotation to the north and south of the azimuthal wind shear track and higher UH probabilities associated with the observed rotation in the northeastern part of the subdomain (Fig. 13). Also, Cyc1+Cyc15’s swaths of UH probabilities and intensities are more precise and closer to the azimuthal wind shear track than PAR15Cyc15’s UH swath, which ends up farther to the south (Fig. 13). By assimilating more PAR volumes earlier in the DA period, Cyc1+Cyc15 subjectively and objectively outperforms PAR15Cyc15. These results demonstrate the potential benefits of adaptive DA cycling intervals to WoFS using next generation PAR observations.
As in Fig. 9, but for Cyc1+Cyc15 and PAR15Cyc15.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
c. Asynchronous DA experiment
While the differences in observation-space diagnostics for PAR1Cyc5 and PAR5Cyc5 are negligible (not shown), some differences do exist among experiments’ reflectivity and 2–5-km UH analyses and forecasts. First, ensemble analyses of 2 km MSL reflectivity from PAR1Cyc5 have less spurious convection on the eastern edge of the El Reno storm, as indicated by initially smaller reflectivity probabilities (Figs. 14b,c). Also, PAR1Cyc5 better forecasts the gap between the El Reno storm and the storm to the north at later forecast times (Figs. 14g,h,l,m). Objectively, PAR1Cyc5 has slightly better eFSS values at all scales (Fig. 15a). Also, PAR1Cyc5’s eFSS values reach FSSref at smaller scales than PAR5Cyc5 at most forecast times (Fig. 15b). In particular, PAR1Cyc5’s forecast launched from 2300 UTC outperforms PAR5Cyc5’s forecast at all times after 2310 UTC (Fig. 15b), which generally agrees with the subjective evaluation.
As in Fig. 5, but for PAR1Cyc5 and PAR5Cyc5.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
As in Fig. 7, but for PAR1Cyc5 and PAR5Cyc5.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
For 2–5-km UH, PAR1Cyc5 and PAR5Cyc5’s forecasts are again mostly similar; however, some notable differences do exist. For example, 1-h forecasts launched from 2300 UTC reveal PAR1Cyc5 results in UH probability and intensity swaths being spatially more centered on the azimuthal wind shear track as indicated by higher probabilities (i.e., >~80%) covering more of the observed rotation track area (Fig. 16). For forecasts initialized at 2230, 2245, and 2300 UTC, PAR1Cyc5’s UH forecasts objectively perform slightly better than PAR5Cyc5’s UH forecasts with larger CSI values and smaller biases (Fig. 17).
As in Fig. 9, but for PAR1Cyc5 and PAR5Cyc5.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
As in Fig. 10, but for PAR1Cyc5 and PAR5Cyc5.
Citation: Weather and Forecasting 35, 1; 10.1175/WAF-D-19-0165.1
Overall, the differences are minimal between the PAR1Cyc5 and PAR5Cyc5 experiments, but asynchronously assimilating additional PAR volumes using the same DA cycling interval with 4DEnSRF does generally improve forecasts of the El Reno storm and surrounding areas. Asynchronous DA is a potential way to improve analyses and forecasts without having to frequently stop the model to assimilate ~1-min PAR volumetric data.
5. Summary and discussion
The NOAA NSSL is actively developing 1) the PAR technologies with dual-polarization capability as a replacement for the aging WSR-88D network and 2) the WoF DA and prediction system to provide NWS forecasters with the NWP model-based probabilistic guidance needed to extend lead times for severe thunderstorm and associated hazards with reduced false alarms. Needless to say, it is critical to evaluate the impact of PAR observations on WoFS. The NWRT PAR collected observations from the 31 May 2013 El Reno tornadic supercell event, and those frequent volumetric data provided an opportunity to determine the optimal temporal frequency of PAR observations for an experimental version of WoFS at 1-km horizontal grid spacing. Those data were used to conduct synchronous and asynchronous EnSRF DA experiments using different DA cycling intervals and PAR volume scan frequencies to produce analyses and forecasts of the El Reno storm. The ensemble forecasts of reflectivity and UH were then assessed with subjective evaluations alongside neighborhood- and object-based verification techniques.
Results from the synchronous DA experiments showed that more frequently assimilating PAR data can more quickly spin up storms and suppress spurious convection in analyses. Specifically, assimilating PAR volumetric data every 1 min produces better analyses and forecasts of the El Reno storm than assimilating PAR data less frequently at 3-, 5-, and 15-min intervals. Also, the improvements going from a 1- to 3- to 5-min cycling interval were smaller than going from a 5- to 15-min cycling interval. Unlike the shorter cycling interval experiments, the longer 15-min cycling interval experiment, which is what is used in the current experimental WoFS, was not able to remove most of the spurious convection within 75 min of DA cycling. Additionally, the longer cycling interval resulted in the El Reno storm propagating too far to the south due to spurious convection and overly strong cold pool. Except for PAR15Cyc15, the forecasts of UH objects began to substantially improve at earlier times for shorter DA cycling intervals. The results from these experiments can likely be generalized to conclude that more frequent DA cycling can lead to more accurate analyses and forecasts at longer lead times.
All of these experiments began with less-than-ideal background forecasts with large amounts of spurious convection owing to no prior radar DA, so the changes in forecast skill among the experiments are in part attributable to suppressing this spurious convection. The current experimental WoFS is initialized from 1-h forecasts provided by the High-Resolution Rapid Refresh Ensemble (Dowell et al. 2016), so spurious convection and storm phase errors in the background forecasts are a realistic concern for the real-time experimental WoFS. Even so, future work will quantify the relative impacts of frequent PAR DA cycling on spinning up storms and suppressing spurious convection.
As Cyc1+Cyc15 demonstrated, using shorter cycling intervals when convection is developing or quickly evolving before switching to a longer cycling interval to maintain current convection is a way to substantially improve analyses and forecasts at longer lead times. The result from this experiment suggests that the adaptive cycling intervals could potentially be beneficial to storm-scale DA systems with noncontinuous cycling like the WoFS. Also, adaptive cycling intervals are computationally less expensive (Table 2) and are a potential solution to any ensemble spread or imbalance issues since frequent DA cycling would be used less often and only in areas where it would have the greatest impact (e.g., observation targeting; Chang 2014).
Total computational costs (core hours) during the 75-min period of DA cycling for each experiment. For reference, the five forecasts for each experiment consumed an estimated combined total of 7200 core hours.
Asynchronously assimilating more frequent PAR observations using 4DEnSRF results in only marginal improvements. Additional experiments not shown in this study were conducted using 4DEnSRF with 5- and 15-min cycling intervals, various temporal frequencies of PAR data, and different time localizations. The results from these additional experiments were similar to the asynchronous experiment shown in this study; synchronous and asynchronous experiments produce analyses and forecasts more similar to each other at the same cycling interval than assimilating a similar number of PAR volumes at different cycling intervals (e.g., PAR1Cyc5 is more similar to PAR5Cyc5 than PAR1Cyc1). Also, PAR1Cyc5’s computational costs are more similar to PAR1Cyc1 than PAR5Cyc5 (Table 2). Based on these findings, a future study is needed to try and understand why PAR1Cyc5 provides little benefit over PAR5Cyc5 at these spatial and temporal scales. It is likely PAR1Cyc5’s marginal improvement over PAR5Cyc5 is due to suboptimal covariance inflation in 4DEnSRF. Even so, until further testing with asynchronous DA results in substantially better analyses and forecasts than the current configurations, more frequent PAR observations will likely have a larger beneficial impact on a storm-scale DA system, such as the WoFS, using shorter cycling intervals with synchronous DA.
The current version of the WoFS (https://wof.nssl.noaa.gov/realtime/) that runs in real time for NOAA testbed experiments uses a coarser 3-km horizontal grid spacing and a longer 15-min DA cycling frequency (e.g., Wheatley et al. 2015; Skinner et al. 2018; Wilson et al. 2019) due to computational constraints. In addition to WSR-88D radar observations, the GOES-16 and GOES-17 geostationary platforms provide observations with approximately kilometer resolution every 5 min. Therefore, with an exponential increase in computational power and availability of high temporal and spatial-resolution observing platforms, WoFS will likely run at smaller grid spacings to resolve the finer-scale details of convection (Bryan et al. 2003) in the future.
The results from this study have a direct implication on the design of the next version of the experimental WoFS. This study demonstrates that an experimental WoFS at 1-km grid spacing with 1-min DA cycling can spin up storms faster in analyses while suppressing spurious convection lending itself to the use of frequent (~1 min) PAR volumetric data for the next generation of the WoFS. To save computational resources, the WoFS could incorporate an adaptive cycling interval technique to assimilate more frequent PAR observations only when necessary. For example, when maintaining slowly evolving convection in analyses and forecasts, less frequent PAR observations would be sufficient. However, when accurate analyses are needed in less time, spurious convection exists in the background forecasts, or convection is developing or quickly evolving, the optimal temporal frequency of PAR observations for storm-scale DA would only be limited by computational resources. Therefore, the optimal temporal frequency of PAR volumetric data would be situationally dependent for the WoFS. With the deployment of NSSL’s Advanced Technology Demonstrator (ATD; Stailey and Hondl 2016), the first full-scale, S-band, dual-polarization PAR in Norman, Oklahoma, in 2018, future work will continue to explore the benefit of rapid-scan PAR on an experimental WoFS using a variety of severe weather events.
Acknowledgments
This research is funded by the Spectrum Efficient National Surveillance Radar (SENSR) program through NOAA/Office of Oceanic and Atmospheric Research under NOAA–University of Oklahoma Cooperative Agreement NA11OAR4320072, U.S. Department of Commerce. Most of the computing for this project was performed at the OU Supercomputing Center for Education and Research (OSCER) at the University of Oklahoma (OU). The authors also acknowledge the Texas Advanced Computing Center (TACC; http://www.tacc.utexas.edu) at the University of Texas at Austin for providing HPC resources that have contributed to the research results reported within this paper. The authors thank Mark Weber and Kurt Hondl for their support and guidance in conducting the research. The authors also thank Charles Kuster for helping with the processing of the NWRT PAR data, Anthony Reinhart for his insight into the MRMS data, and Pamela Heinselman for her helpful suggestions. The authors thank the three anonymous reviewers for their comments and suggestions, which improved the manuscript.
REFERENCES
Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129, 2884–2903, https://doi.org/10.1175/1520-0493(2001)129<2884:AEAKFF>2.0.CO;2.
Anderson, J. L., and N. Collins, 2007: Scalable implementations of ensemble filter algorithms for data assimilation. J. Atmos. Oceanic Technol., 24, 1452–1463, https://doi.org/10.1175/JTECH2049.1.
Benjamin, S. G., and Coauthors, 2016: A North American hourly assimilation and model forecast cycle: The Rapid Refresh. Mon. Wea. Rev., 144, 1669–1694, https://doi.org/10.1175/MWR-D-15-0242.1.
Bluestein, H. B., J. C. Snyder, and J. B. Houser, 2015: A multiscale overview of the El Reno, Oklahoma, tornadic supercell of 31 May 2013. Wea. Forecasting, 30, 525–552, https://doi.org/10.1175/WAF-D-14-00152.1.
Bowden, K. A., and P. L. Heinselman, 2016: A qualitative analysis of NWS forecasters’ use of phased-array radar data during severe hail and wind events. Wea. Forecasting, 31, 43–55, https://doi.org/10.1175/WAF-D-15-0089.1.
Bowden, K. A., P. L. Heinselman, D. M. Kingfield, and R. P. Thomas, 2015: Impacts of phased-array radar data on forecaster performance during severe hail and wind events. Wea. Forecasting, 30, 389–404, https://doi.org/10.1175/WAF-D-14-00101.1.
Bryan, G. H., J. C. Wyngaard, and J. M. Fritsch, 2003: Resolution requirements for the simulation of deep moist convection. Mon. Wea. Rev., 131, 2394–2416, https://doi.org/10.1175/1520-0493(2003)131<2394:RRFTSO>2.0.CO;2.
Chang, W., 2014: Convective-scale radar data assimilation and adaptive radar observation with the ensemble Kalman filter. Ph.D. thesis, Department of Atmospheric and Oceanic Sciences, McGill University, 154 pp.
Davis, C., B. Brown, and R. Bullock, 2006a: Object-based verification of precipitation forecasts. Part I: Methods and application to mesoscale rain areas. Mon. Wea. Rev., 134, 1772–1784, https://doi.org/10.1175/MWR3145.1.
Davis, C., B. Brown, and R. Bullock, 2006b: Object-based verification of precipitation forecasts. Part II: Application to convective rain systems. Mon. Wea. Rev., 134, 1785–1795, https://doi.org/10.1175/MWR3146.1.
Dowell, D. C., and L. J. Wicker, 2009: Additive noise for storm-scale ensemble data assimilation. J. Atmos. Oceanic Technol., 26, 911–927, https://doi.org/10.1175/2008JTECHA1156.1.
Dowell, D. C., L. J. Wicker, and C. Snyder, 2011: Ensemble Kalman filter assimilation of radar observations of the 8 May 2003 Oklahoma City supercell: Influences of reflectivity observations on storm-scale analyses. Mon. Wea. Rev., 139, 272–294, https://doi.org/10.1175/2010MWR3438.1.
Dowell, D. C., and Coauthors, 2016: Development of a High-Resolution Rapid Refresh Ensemble (HRRRE) for severe weather forecasting. 28th Conf. on Severe Local Storms, Portland, OR, Amer. Meteor. Soc., 8B.2, https://ams.confex.com/ams/28SLS/webprogram/Paper301555.html.
Duc, L., K. Saito, and H. Seko, 2013: Spatial–temporal fractions verification for high-resolution ensemble forecasts. Tellus, 65A, 18171, https://doi.org/10.3402/tellusa.v65i0.18171.
Ebert, E. E., 2001: Ability of a poor man’s ensemble to predict the probability and distribution of precipitation. Mon. Wea. Rev., 129, 2461–2480, https://doi.org/10.1175/1520-0493(2001)129<2461:AOAPMS>2.0.CO;2.
Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99, 10 143–10 162, https://doi.org/10.1029/94JC00572.
Forsyth, D. E., and Coauthors, 2005: The National Weather Radar Testbed (phased array). 32nd Conf. on Radar Meteorology, Albuquerque, NM, Amer. Meteor. Soc., 12R.3, https://ams.confex.com/ams/32Rad11Meso/techprogram/paper_96377.htm.
Gaspari, G., and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteor. Soc., 125, 723–757, https://doi.org/10.1002/qj.49712555417.
Heinselman, P. L., and S. M. Torres, 2011: High-temporal-resolution capabilities of the National Weather Radar Testbed Phased-Array Radar. J. Appl. Meteor. Climatol., 50, 579–593, https://doi.org/10.1175/2010JAMC2588.1.
Hu, M., H. Shao, D. Stark, K. Newman, and C. Zhou, 2015a: Gridpoint Statistical Interpolation (GSI) user’s guide for version 3.4. Developmental Testbed Center, 143 pp., http://www.dtcenter.org/com-GSI/users/docs/index.php.
Hu, M., H. Shao, D. Stark, K. Newman, and C. Zhou, 2015b: Ensemble Kalman Filter (EnKF) user’s guide for version 1.0. Developmental Testbed Center, 55 pp., http://www.dtcenter.org/EnKF/users/docs/index.php.
Jones, T. A., K. Knopfmeier, D. Wheatley, G. Creager, P. Minnis, and R. Palikondo, 2016: Storm-scale data assimilation and ensemble forecasting with the NSSL Experimental Warn-on-Forecast System. Part II: Combined radar and satellite data experiments. Wea. Forecasting, 31, 297–327, https://doi.org/10.1175/WAF-D-15-0107.1.
Jones, T. A., P. Skinner, K. Knopfmeier, E. Mansell, P. Minnis, R. Palikonda, and W. Smith, 2018: Comparison of cloud microphysics schemes in a Warn-on-Forecast System using synthetic satellite objects. Wea. Forecasting, 33, 1681–1708, https://doi.org/10.1175/WAF-D-18-0112.1.
Kain, J. S., and Coauthors, 2008: Some practical considerations regarding horizontal resolution in the first generation of operational convection-allowing NWP. Wea. Forecasting, 23, 931–952, https://doi.org/10.1175/WAF2007106.1.
Kuster, C. M., P. L. Heinselman, and M. Austin, 2015: 31 May 2013 El Reno tornadoes: Advantages of rapid-scan phased-array radar data from a warning forecaster’s perspective. Wea. Forecasting, 30, 933–956, https://doi.org/10.1175/WAF-D-14-00142.1.
Lakshmanan, V., T. M. Smith, K. Hondl, G. J. Stumpf, and A. Witt, 2006: A real-time, three-dimensional, rapidly updating, heterogeneous radar merger technique for reflectivity, velocity, and derived products. Wea. Forecasting, 21, 802–823, https://doi.org/10.1175/WAF942.1.
Lakshmanan, V., T. M. Smith, G. J. Stumpf, and K. D. Hondl, 2007: The Warning Decision Support System—Integrated information. Wea. Forecasting, 22, 596–612, https://doi.org/10.1175/WAF1009.1.
Lange, H., and G. C. Craig, 2014: The impact of data assimilation length scales on analysis and prediction of convective storms. Mon. Wea. Rev., 142, 3781–3808, https://doi.org/10.1175/MWR-D-13-00304.1.
Lawson, J. R., J. S. Kain, N. Yussouf, D. C. Dowell, D. M. Wheatley, K. H. Knopfmeier, and T. A. Jones, 2018: Advancing from convection-allowing NWP to Warn-on-Forecast: Evidence of progress. Wea. Forecasting, 33, 599–607, https://doi.org/10.1175/WAF-D-17-0145.1.
Lei, T., M. Xue, T. Y. Yu, and M. Teshiba, 2007: Study on the optimal scanning strategies of phase-array radar through ensemble Kalman filter assimilation of simulated data. 33rd Int. Conf. on Radar Meteorology, Cairns, Australia, Amer. Meteor. Soc., P7.1, http://ams.confex.com/ams/pdfpapers/124022.pdf.
Mansell, E. R., C. L. Ziegler, and E. C. Bruning, 2010: Simulated electrification of a small thunderstorm with two–moment bulk microphysics. J. Atmos. Sci., 67, 171–194, https://doi.org/10.1175/2009JAS2965.1.
Miller, M. L., V. Lakshmanan, and T. Smith, 2013: An automated method for depicting mesocyclone paths and intensities. Wea. Forecasting, 28, 570–585, https://doi.org/10.1175/WAF-D-12-00065.1.
NOAA, 2014: Service assessment: May 2013 Oklahoma tornadoes and flash flooding. National Weather Service, 42 pp. + appendixes, https://www.weather.gov/media/publications/assessments/13oklahoma_tornadoes.pdf.
NOAA/NWS, 2013: The May 31–June 1, 2013 tornado and flash flooding event. NOAA/NWS, accessed 22 July 2019, https://www.weather.gov/oun/events-20130531.
Potvin, C. K., and M. L. Flora, 2015: Sensitivity of idealized supercell simulations to horizontal grid spacing: Implications for Warn-on-Forecast. Mon. Wea. Rev., 143, 2998–3024, https://doi.org/10.1175/MWR-D-14-00416.1.
Roberts, N. M., and H. W. Lean, 2008: Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Mon. Wea. Rev., 136, 78–97, https://doi.org/10.1175/2007MWR2123.1.
Roebber, P. J., 2009: Visualizing multiple measures of forecast quality. Wea. Forecasting, 24, 601–608, https://doi.org/10.1175/2008WAF2222159.1.
Schenkman, A. D., M. Xue, A. Shapiro, K. Brewster, and J. Gao, 2011: Impact of CASA radar and Oklahoma Mesonet data assimilation on the analysis and prediction of tornadic mesovortices in an MCS. Mon. Wea. Rev., 139, 3422–3445, https://doi.org/10.1175/MWR-D-10-05051.1.
Skamarock, W. C., and Coauthors, 2008: A description of the Advanced Research WRF version 3. NCAR Tech. Note NCAR/TN-475+STR, 113 pp., https://doi.org/10.5065/D68S4MVH.
Skinner, P. S., L. Wicker, D. M. Wheatley, and K. H. Knopfmeier, 2016: Application of two spatial verification methods to ensemble forecasts of low-level rotation. Wea. Forecasting, 31, 713–735, https://doi.org/10.1175/WAF-D-15-0129.1.
Skinner, P. S., and Coauthors, 2018: Object-based verification of a prototype Warn-on-Forecast system. Wea. Forecasting, 33, 1225–1250, https://doi.org/10.1175/WAF-D-18-0020.1.
Smith, T. M., and K. L. Elmore, 2004: The use of radial velocity derivatives to diagnose rotation and divergence. 11th Conf. on Aviation, Range, and Aerospace, Hyannis, MA, Amer. Meteor. Soc., P5.6, https://ams.confex.com/ams/pdfpapers/81827.pdf.
Smith, T. M., and Coauthors, 2016: Multi-Radar Multi-Sensor (MRMS) severe weather and aviation products: Initial operating capabilities. Bull. Amer. Meteor. Soc., 97, 1617–1630, https://doi.org/10.1175/BAMS-D-14-00173.1.
Snyder, J. C., and H. B. Bluestein, 2014: Some considerations for the use of high-resolution mobile radar data in tornado intensity determination. Wea. Forecasting, 29, 799–827, https://doi.org/10.1175/WAF-D-14-00026.1.
Sobash, R. A., and D. J. Stensrud, 2015: Assimilating surface mesonet observations with the EnKF to improve ensemble forecasts of convection initiation on 29 May 2012. Mon. Wea. Rev., 143, 3700–3725, https://doi.org/10.1175/MWR-D-14-00126.1.
Sobash, R. A., C. S. Schwartz, G. S. Romine, and M. L. Weisman, 2019: Next-day prediction of tornadoes using convection-allowing models with 1-km horizontal grid spacing. Wea. Forecasting, 34, 1117–1135, https://doi.org/10.1175/WAF-D-19-0044.1.
Stailey, J. E., and K. D. Hondl, 2016: Multifunction phased array radar for aircraft and weather surveillance. Proc. IEEE, 104, 649–659, https://doi.org/10.1109/JPROC.2015.2491179.
Stensrud, D. J., and Coauthors, 2009: Convective-scale warn-on-forecast system: A vision for 2020. Bull. Amer. Meteor. Soc., 90, 1487–1499, https://doi.org/10.1175/2009BAMS2795.1.
Stensrud, D. J., and Coauthors, 2013: Progress and challenges with Warn-on-Forecast. Atmos. Res., 123, 2–16, https://doi.org/10.1016/j.atmosres.2012.04.004.
Supinie, T. A., N. Yussouf, Y. Jung, M. Xue, J. Cheng, and S. Wang, 2017: Comparison of the analyses and forecasts of a tornadic supercell storm from assimilating phased-array radar and WSR-88D observations. Wea. Forecasting, 32, 1379–1401, https://doi.org/10.1175/WAF-D-16-0159.1.
Tewari, M., and Coauthors, 2004: Implementation and verification of the unified NOAH land surface model in the WRF model. 20th Conf. on Weather Analysis and Forecasting/16th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., 14.2A, https://ams.confex.com/ams/pdfpapers/69061.pdf.
Thompson, G., P. R. Field, R. M. Rasmussen, and W. D. Hall, 2008: Explicit forecasts of winter precipitation using an improved bulk microphysics scheme. Part II: Implementation of a new snow parameterization. Mon. Wea. Rev., 136, 5095–5115, https://doi.org/10.1175/2008MWR2387.1.
Toth, Z., Y. Zhu, and R. Wobus, 2004: March 2004 upgrades of the NCEP global ensemble forecast system. NOAA/NCEP/EMC. Accessed 2019, http://www.emc.ncep.noaa.gov/gmb/ens/ens_imp_news.html.
VandenBerg, M. A., M. C. Coniglio, and A. J. Clark, 2014: Comparison of next-day convection-allowing forecasts of storm motion on 1- and 4-km grids. Wea. Forecasting, 29, 878–893, https://doi.org/10.1175/WAF-D-14-00011.1.
Wakimoto, R. M., N. T. Atkins, K. M. Butler, H. B. Bluestein, K. Thiem, J. Snyder, and J. Houser, 2015: Photogrammetric analysis of the 2013 El Reno tornado combined with mobile X-band polarimetric radar data. Mon. Wea. Rev., 143, 2657–2683, https://doi.org/10.1175/MWR-D-15-0034.1.
Wakimoto, R. M., and Coauthors, 2016: Aerial damage survey of the 2013 El Reno tornado combined with mobile radar data. Mon. Wea. Rev., 144, 1749–1776, https://doi.org/10.1175/MWR-D-15-0367.1.
Wang, S., M. Xue, and J. Min, 2013: A four-dimensional asynchronous ensemble square-root filter (4DEnSRF) algorithm and tests with simulated radar data. Quart. J. Roy. Meteor. Soc., 139, 805–819, https://doi.org/10.1002/qj.1987.
Wang, Y., Y. Jung, T. A. Supinie, and M. Xue, 2013: A hybrid MPI-OpenMP parallel algorithm and performance analysis for an ensemble square root filter designed for multiscale observations. J. Atmos. Oceanic Technol., 30, 1382–1397, https://doi.org/10.1175/JTECH-D-12-00165.1.
Weber, M. E., J. Y. N. Cho, J. S. Herd, J. M. Flavin, W. E. Benner, and G. S. Torok, 2007: The next-generation multimission U.S. surveillance radar network. Bull. Amer. Meteor. Soc., 88, 1739–1751, https://doi.org/10.1175/BAMS-88-11-1739.
Weber, M. E., J. Y. Cho, and H. G. Thomas, 2017: Command and control for multifunction phased array radar. IEEE Trans. Geosci. Remote Sens., 55, 5899–5912, https://doi.org/10.1109/TGRS.2017.2716935.
Wei, M., Z. Toth, R. Wobus, and Y. Zhu, 2008: Initial perturbations based on the ensemble transform (ET) technique in the NCEP global operational forecast system. Tellus, 60A, 62–79, https://doi.org/10.1111/j.1600-0870.2007.00273.x.
Wheatley, D. M., K. H. Knopfmeier, T. A. Jones, and G. J. Creager, 2015: Storm-scale data assimilation and ensemble forecasting with the NSSL experimental Warn-on-Forecast system. Part I: Radar data experiments. Wea. Forecasting, 30, 1795–1817, https://doi.org/10.1175/WAF-D-15-0043.1.
Whitaker, J. S., and T. M. Hamill, 2002: Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 130, 1913–1924, https://doi.org/10.1175/1520-0493(2002)130<1913:EDAWPO>2.0.CO;2.
Whitaker, J. S., and T. M. Hamill, 2012: Evaluating methods to account for system errors in ensemble data assimilation. Mon. Wea. Rev., 140, 3078–3089, https://doi.org/10.1175/MWR-D-11-00276.1.
Wilson, K. A., P. L. Heinselman, C. M. Kuster, D. M. Kingfield, and Z. Kang, 2017: Forecaster performance and workload: Does radar update time matter? Wea. Forecasting, 32, 253–274, https://doi.org/10.1175/WAF-D-16-0157.1.
Wilson, K. A., P. L. Heinselman, P. S. Skinner, J. J. Choate, and K. E. Klockow-McClain, 2019: Meteorologists interpretations of storm scale ensemble-based forecast guidance. Wea. Climate Soc., 11, 337–354, https://doi.org/10.1175/WCAS-D-18-0084.1.
Wurman, J., K. Kosiba, P. Robinson, and T. Marshall, 2014: The role of multiple-vortex tornado structure in causing storm researcher fatalities. Bull. Amer. Meteor. Soc., 95, 31–45, https://doi.org/10.1175/BAMS-D-13-00221.1.
Xue, M., D.-H. Wang, J.-D. Gao, K. Brewster, and K. K. Droegemeier, 2003: The Advanced Regional Prediction System (ARPS), storm-scale numerical weather prediction and data assimilation. Meteor. Atmos. Phys., 82, 139–170, https://doi.org/10.1007/s00703-001-0595-6.
Xue, M., M. Tong, and K. K. Droegemeier, 2006: An OSSE framework based on the ensemble square-root Kalman filter for evaluating impact of data from radar networks on thunderstorm analysis and forecast. J. Atmos. Oceanic Technol., 23, 46–66, https://doi.org/10.1175/JTECH1835.1.
Yussouf, N., and D. J. Stensrud, 2010: Impact of phased-array radar observations over a short assimilation period: Observing system simulation experiments using an ensemble Kalman filter. Mon. Wea. Rev., 138, 517–538, https://doi.org/10.1175/2009MWR2925.1.
Yussouf, N., and K. H. Knopfmeier, 2019: Application of Warn-on-Forecast system for flash-flood producing heavy convective rainfall events. Quart. J. Roy. Meteor. Soc., 145, 2385–2403, https://doi.org/10.1002/qj.3568.
Yussouf, N., D. C. Dowell, L. J. Wicker, K. H. Knopfmeier, and D. M. Wheatley, 2015: Storm-scale data assimilation and ensemble forecasts for the 27 April 2011 severe weather outbreak in Alabama. Mon. Wea. Rev., 143, 3044–3066, https://doi.org/10.1175/MWR-D-14-00268.1.
Yussouf, N., J. S. Kain, and A. J. Clark, 2016: Short-term probabilistic forecasts of the 31 May 2013 Oklahoma tornado and flash flood event using a continuous-update-cycle storm-scale ensemble system. Wea. Forecasting, 31, 957–983, https://doi.org/10.1175/WAF-D-15-0160.1.
Zhang, F., C. Snyder, and J. Sun, 2004: Impacts of initial estimate and observation availability on convective-scale data assimilation with an ensemble Kalman filter. Mon. Wea. Rev., 132, 1238–1253, https://doi.org/10.1175/1520-0493(2004)132<1238:IOIEAO>2.0.CO;2.
Zrnić, D. S., and Coauthors, 2007: Agile-beam phased array radar for weather observations. Bull. Amer. Meteor. Soc., 88, 1753–1766, https://doi.org/10.1175/BAMS-88-11-1753.