Simulation of Polarimetric Radar Variables from 2013 CAPS Spring Experiment Storm-Scale Ensemble Forecasts and Evaluation of Microphysics Schemes

Bryan J. Putnam Center for Analysis and Prediction of Storms, and Advanced Radar Research Center, and School of Meteorology, University of Oklahoma, Norman, Oklahoma

Search for other papers by Bryan J. Putnam in
Current site
Google Scholar
PubMed
Close
,
Ming Xue Center for Analysis and Prediction of Storms, and School of Meteorology, University of Oklahoma, Norman, Oklahoma

Search for other papers by Ming Xue in
Current site
Google Scholar
PubMed
Close
,
Youngsun Jung Center for Analysis and Prediction of Storms, University of Oklahoma, Norman, Oklahoma

Search for other papers by Youngsun Jung in
Current site
Google Scholar
PubMed
Close
,
Guifu Zhang Advanced Radar Research Center, and School of Meteorology, University of Oklahoma, Norman, Oklahoma

Search for other papers by Guifu Zhang in
Current site
Google Scholar
PubMed
Close
, and
Fanyou Kong Center for Analysis and Prediction of Storms, Norman, Oklahoma

Search for other papers by Fanyou Kong in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

Polarimetric radar variables are simulated from members of the 2013 Center for Analysis and Prediction of Storms (CAPS) Storm-Scale Ensemble Forecasts (SSEF) with varying microphysics (MP) schemes and compared with observations. The polarimetric variables provide information on hydrometeor types and particle size distributions (PSDs), neither of which can be obtained through reflectivity (Z) alone. The polarimetric radar simulator pays close attention to how each MP scheme [including single- (SM) and double-moment (DM) schemes] treats hydrometeor types and PSDs. The recent dual-polarization upgrade to the entire WSR-88D network provides nationwide polarimetric observations, allowing for direct evaluation of the simulated polarimetric variables.

Simulations for a mesoscale convective system (MCS) and supercell cases are examined. Five different MP schemes—Thompson, DM Milbrandt and Yau (MY), DM Morrison, WRF DM 6-category (WDM6), and WRF SM 6-category (WSM6)—are used in the ensemble forecasts. Forecasts using the partially DM Thompson and fully DM MY and Morrison schemes better replicate the MCS structure and stratiform precipitation coverage, as well as supercell structure compared to WDM6 and WSM6. Forecasts using the MY and Morrison schemes better replicate observed polarimetric signatures associated with size sorting than those using the Thompson, WDM6, and WSM6 schemes, in which such signatures are either absent or occur at abnormal locations. Several biases are suggested in these schemes, including too much wet graupel in MY, Morrison, and WDM6; a small raindrop bias in WDM6 and WSM6; and the underforecast of liquid water content in regions of pure rain for all schemes.

Corresponding author address: Ming Xue, Center for Analysis and Prediction of Storms, University of Oklahoma, 120 David Boren Blvd., Norman, OK 73072. E-mail: mxue@ou.edu

Abstract

Polarimetric radar variables are simulated from members of the 2013 Center for Analysis and Prediction of Storms (CAPS) Storm-Scale Ensemble Forecasts (SSEF) with varying microphysics (MP) schemes and compared with observations. The polarimetric variables provide information on hydrometeor types and particle size distributions (PSDs), neither of which can be obtained through reflectivity (Z) alone. The polarimetric radar simulator pays close attention to how each MP scheme [including single- (SM) and double-moment (DM) schemes] treats hydrometeor types and PSDs. The recent dual-polarization upgrade to the entire WSR-88D network provides nationwide polarimetric observations, allowing for direct evaluation of the simulated polarimetric variables.

Simulations for a mesoscale convective system (MCS) and supercell cases are examined. Five different MP schemes—Thompson, DM Milbrandt and Yau (MY), DM Morrison, WRF DM 6-category (WDM6), and WRF SM 6-category (WSM6)—are used in the ensemble forecasts. Forecasts using the partially DM Thompson and fully DM MY and Morrison schemes better replicate the MCS structure and stratiform precipitation coverage, as well as supercell structure compared to WDM6 and WSM6. Forecasts using the MY and Morrison schemes better replicate observed polarimetric signatures associated with size sorting than those using the Thompson, WDM6, and WSM6 schemes, in which such signatures are either absent or occur at abnormal locations. Several biases are suggested in these schemes, including too much wet graupel in MY, Morrison, and WDM6; a small raindrop bias in WDM6 and WSM6; and the underforecast of liquid water content in regions of pure rain for all schemes.

Corresponding author address: Ming Xue, Center for Analysis and Prediction of Storms, University of Oklahoma, 120 David Boren Blvd., Norman, OK 73072. E-mail: mxue@ou.edu

1. Introduction

The national WSR-88D S-band weather radar network has completed its polarimetric upgrade, providing unprecedented polarimetric radar variable measurements over the CONUS (ROC 2013). The polarimetric radar variables provide additional information about the cloud hydrometeor types and their particle size distributions (PSDs) compared to reflectivity (Z), in particular information on hydrometeor size and diversity. The variables include 1) differential reflectivity (ZDR) that is sensitive to hydrometeor shape, orientation, and phase; 2) specific differential phase (KDP) that is sensitive to rainwater content/rain rate; and 3) cross-correlation coefficient (ρhv) that is sensitive to diverse and mixed-phase hydrometeors (Bringi and Chandrasekar 2001). Common dynamical and microphysical processes lead to patterns in these variables that occur at specific locations and in specific circumstances within convective storms, referred to as polarimetric signatures (Kumjian and Ryzhkov 2008). For example, there is a relative ZDR maximum along the right-forward flank of supercells as a result of hydrometeor size sorting, known as the ZDR arc. In mesoscale convective systems (MCSs), high ZDR is observed on the leading edge of the convective line because of the size sorting of larger drops that fall ahead of the system (Park et al. 2009).

The hydrometeor variables in microphysics (MP) schemes of numerical weather prediction (NWP) models, such as mixing ratio (q), are typically not directly observed. One way to evaluate the model prediction of hydrometeor fields and the MP parameterization schemes is to simulate polarimetric variables from the model output and compare them with observations. The model state variables, including MP variables, are connected to the observed polarimetric fields by the so-called polarimetric radar data simulator (PRDS; Jung et al. 2008a; Jung et al. 2010, hereafter JXZ10), or the observation operators in data assimilation terminology. These operators are derived from scattering calculations of polarized radar radio waves by hydrometeor particles within each radar sampling volume.

Most MP schemes represent hydrometeor PSDs in bulk form using the simplified gamma distribution,
e1
which defines the number of particles of hydrometeor x with diameter D in a unit volume (Ulbrich 1983; Milbrandt and Yau 2005a). Three free parameters govern the distribution: 1) the slope parameter , 2) the intercept parameter , and 3) the shape parameter . MP schemes can be broadly categorized by the numbers of these free parameters that they derive from predicted microphysical variables for each species. For example, q is proportional to the third PSD moment (mass) and is used to solve for . Single-moment (SM), double-moment (DM), and triple-moment (TM) schemes predict one, two, and three moments of the PSD and can therefore determine one, two, or three of the PSD parameters, respectively. Parameters that are not derived from predicted variables are either diagnosed or set as constant. Another significant feature of a given MP scheme is the number of hydrometeor species included. Five categories are most commonly considered in ice MP schemes: cloud water (c), cloud ice (i), rainwater (r), snow (s), and graupel (g) or hail (h), and some but relatively few schemes [e.g., the Milbrandt and Yau (MY) scheme (Milbrandt and Yau 2005b)] include graupel and hail as separate species.

It is important that the observation operators developed for a PRDS are consistent with the MP scheme so that the simulated variables reflect the model microphysical state and dynamical processes. Increasing the number of model variables predicted (e.g., moving from an SM to a DM scheme) increases the amount of predicted microphysical information that can and should be used in the operators. Some schemes, including the Thompson (Thompson et al. 2008) and WDM6 (Lim and Hong 2010) schemes are partially double moment, predicting a second moment for rain (number concentration, Ntr) but only one moment for the other hydrometeor species. Though most SM and DM schemes set by default, resulting in an exponential distribution, WDM6 uses for rain and the Thompson scheme uses a combined exponential and gamma distribution for snow.

Limitations of MP schemes may preclude the model from replicating certain polarimetric signatures and highlight microphysical state differences. Some current schemes, including the Thompson, WSM6 (Hong and Lim 2006), WDM6, and Morrison (Morrison et al. 2005; Morrison et al. 2009) schemes, contain a graupel category but not hail.1 In a supercell simulation experiment by Johnson et al. (2016) using these schemes, the hail signature in the forward-flank downdraft, a decrease in ZDR associated with large, dry hail (Kumjian and Ryzhkov 2008), was not replicated by those schemes that only include a graupel category due to the small size of and limited amount of graupel present near the surface and the associated high rainwater content. Additionally, Wacker and Seifert (2001) and Milbrandt and Yau (2005a) have shown that SM MP schemes cannot represent sedimentation, or size sorting, and thus a DM or higher-order scheme is required to produce polarimetric signatures associated with size sorting (JXZ10; Kumjian and Ryzhkov 2012). Jung et al. (2012) demonstrated that the ZDR arc signature could be replicated with the DM MY scheme but not an SM Lin (Lin et al. 1983) scheme when the states of a supercell are estimated using a cycled ensemble Kalman filter. Putnam et al. (2014) showed for an MCS case that the size sorting of large drops and subsequent increase in ZDR in the convective line compared to the stratiform region could be replicated by the DM MY scheme but not the SM Lin scheme.

Since the spring of 2007, the Center for Analysis and Prediction of Storms (CAPS) at the University of Oklahoma has been producing Storm-Scale Ensemble Forecasts (SSEFs) for the CONUS (Kong et al. 2007; Xue et al. 2007) as part of the NOAA Hazardous Weather Testbed (HWT) Spring Experiment (Weiss et al. 2007; Clark et al. 2012). In the spring of 2013, the SSEF system had a 4-km convection-permitting grid spacing, allowing explicit representation of convective storms (Kong 2013). The system used a variety of MP schemes among its ensemble members, including several that predicted two moments of some of the hydrometeor species within the schemes. One set of special products produced from the ensemble forecast output were simulated polarimetric radar variables using the PRDS developed at CAPS. The availability of polarimetric observations from the upgraded WSR-88D network and the PRDS with the ability to simulate polarimetric variables from a variety of MP schemes provided an unprecedented opportunity to compare and contrast the ability of the various MP schemes commonly used in storm-scale forecasts in reproducing known polarimetric signatures.

The purpose of this paper is to document the real-time implementation of the PRDS, evaluate the simulated polarimetric variables and polarimetric variable forecasts within the SSEF system against WSR-88D polarimetric observations, and to infer the strengths and weaknesses of MP schemes as implemented in the 2013 ensemble. Biases identified with the MP schemes can help the scheme developers to improve their schemes, and help the scheme users to interpret their simulation results in their research. As observed quantities are often more intuitive to forecasters, simulated polarimetric variables can be used by forecasters to monitor and nowcast severe weather when the association of polarimetric signatures with weather events–features is well recognized. Toward that end, knowledge gained on the behavior of the MP schemes and the PRDS can help forecasters better understand the dual-polarization forecast products.

Up to the time of this exercise, CAPS’s PRDS had mainly been used with the MY DM scheme (Jung et al. 2012; Putnam et al. 2014); this effort represents the first time that multiple DM schemes have been evaluated within a common framework in terms of their ability to produce polarimetric radar signatures for real cases. The recent study of Johnson et al. (2016) had a similar goal but it was based on a set of idealized supercell simulations and therefore no real radar observations could be used for comparison. Since significant polarimetric radar signatures are relatively local and isolated within convective systems, simple gridpoint-based evaluation scores typically applied to precipitation forecasts, such as the equitable threat score, are not very revealing, especially when different types of convective systems are mixed together (different types of convective systems tend to produce different kinds of polarimetric signatures in different parts of the systems). Because of the many challenges facing objective evaluations of the forecast of polarimetric signatures, which tend to be highly localized and in the current forecasts contain significant biases, we choose to focus on two cases from the CAPS 2013 Spring Experiment only in this study: one case with MCSs and one case with supercells. Focusing on two cases allows us to perform more detailed subjective evaluations and at the same time access the objective evaluation methods and procedures. Such a study would provide the groundwork for future studies evaluating the MP and PRDS performances over the entire Spring Experiment period.

The rest of this paper is organized as follows. The methodologies, including the general design of the CAPS SSEF the PRDS and MP schemes used, and the quality control of observations, are given in section 2. Section 3 presents evaluation results for the MCS and supercell cases. A summary and our conclusions, reviewing notable trends in polarimetric size sorting signatures and biases in graupel and water content, are given in section 4. Some challenges faced in evaluation are also noted.

2. Methodology

a. Overview of the 2013 CAPS SSEF

The 2013 CAPS SSEF forecasts were run as part of the NOAA HWT Spring Experiment (Kong 2013). Official forecasts began on 6 May 2013 and continued through 7 June 2013. Daily 48-h forecasts initialized at 0000 UTC were run on a CONUS domain using a horizontal grid spacing of 4 km with 51 vertical levels (Fig. 1). Twenty-nine ensemble members were run using three mesoscale NWP models: the WRF-ARW model [version 3.4.1, 26 members; Skamarock et al. (2008)], the U.S. Navy’s COAMPS model [two members; Hodur (1997)], and the CAPS Advanced Regional Prediction System [ARPS, version 5.3, one member; Xue et al. (2003)]. This paper focuses on the WRF-ARW members since the COAMPS and ARPS members used SM MP schemes only.

Fig. 1.
Fig. 1.

Model domain for 2013 CAPS Spring Experiment SSEFs.

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

The 26 WRF-ARW members varied in terms of their initial conditions (ICs), boundary conditions (BCs), and physics packages. The control member IC was obtained by assimilating surface, upper-air, and WSR-88D radar observations using the ARPS 3DVAR and complex cloud analysis system (Xue et al. 2003; Gao et al. 2004; Hu et al. 2006a,b), with the NCEP 12-km NAM (Rogers et al. 2009) 0000 UTC analysis used as the background, and used BCs that were obtained from the 0000 UTC NAM forecast. An additional 11 members used this IC and these BCs while 13 members used this IC and these BCs with added perturbations derived from the 2100 UTC NCEP Short-Range Ensemble Forecasting system (SREF; Du et al. 2006) forecasts. One member was initialized from the NAM analysis directly. For the purpose of investigating the performance of various physics packages in the WRF-ARW model, the subset of members that used the same IC and BCs as the control member differed in their use of land surface, boundary layer, radiation, and MP schemes. Since MP scheme differences are the focus of this study, polarimetric variable simulations are performed for the control and for those members that differed from the control only in their choice of MP scheme (the arw_cn, arw_m20, arw_m21, arw_m22, and arw_m26 members) (Kong 2013). These members used the Noah land surface model (Chen and Dudhia 2001) and the Mellor–Yamada–Janjić boundary layer scheme (MYJ; Mellor and Yamada 1982; Janjić 2002). More details on the MP schemes used are provided in section 2c.

b. Polarimetric simulation and general experiment settings

The PRDS originally developed for ARPS output (Jung et al. 2008a; JXZ10) was adapted and applied to the WRF-ARW output with several different MP schemes. The PRDS calculations include only the rain, snow, graupel, and hail categories, when applicable. Despite the important role that cloud water and cloud ice play in precipitation processes, the radar returns from these hydrometeors are minimal. Important details of the PRDS, including the axis-ratio relation, canting angle of particles, the melting model, and radar scattering amplitudes, are briefly summarized here.

The PRDS operators include complex scattering amplitudes calculated using the T-matrix method (Vivekanandan et al. 1991; Bringi and Chandrasekar 2001) for both rain and ice species via numerical integration over the PSDs. The raindrop axis ratio decreases with diameter based on the relation in Brandes et al. (2002); this ratio is set to 0.75 for hail, graupel, and snow. The mean canting angle for all hydrometeor types is 0° with a standard deviation of 0° for rain, 20° for snow, and ranging from 0° to 60° for hail and graupel depending on the water fraction. Since most MP schemes do not predict mixed-phase hydrometeors, a mixing ratio fraction of wet (melting) snow, wet hail, or wet graupel is considered present when rain (qr) coexists at a particular model grid point with snow (qs), hail (qh), or graupel (qg), creating mixed-phase mixing ratios denoted qrs, qrh, and qrg. The water fraction model used for the mixed phases is described in detail in Jung et al. (2008a), and the water fraction model used during the 2013 CAPS Spring Experiment does not vary across the size spectrum. The density (ρ) of each mixed-phase species increases as the fractional amount of rain increases and the dielectric constant is calculated using the Maxwell-Garnett mixing formula (Maxwell-Garnett 1904). These variables are used in separate calculations of Zrs, Zrh, and Zrg for mixtures, in addition to Zr, Zs, Zg, and Zh, with the log of the sum giving the final simulated Z. A radar wavelength of 107 mm is used to match the WSR-88D S-band network. For reference, from JXZ10, Z is calculated using their Eq. (3), ZDR from the quotient of their Eqs. (3) and (4), and KDP from their Eq. (6).

c. Spring Experiment microphysics schemes

The 2013 SSEF WRF-ARW members used six different MP schemes: the MY, Morrison, Thompson, WDM6, NSSL (Mansell 2010), and WSM6 schemes. The NSSL scheme has not yet been added to the PRDS because its representation of hydrometeor PSDs is considerably more complex than the other schemes. The original PRDS operators were already compatible with the WSM6 and MY schemes. The Morrison scheme follows the same PSD and has the same predicted moments as MY (having either graupel or hail) so it was easily implemented. Modifications were required for the other schemes. The Thompson and WDM6 schemes predict Nt and q for rain but only predict q for the remaining categories as used in the PRDS [prediction of Ntr was added to the Thompson scheme after Thompson et al. (2008)]. WDM6 diagnoses using temperature and uses a fixed value for . WDM6 also uses a fixed shape parameter of 1.0 for αr. The Thompson scheme has been further updated since Thompson et al. (2008) to use temperature and the mean volume diameter of rain to diagnose . The Thompson scheme also deviates from the typical representation of the bulk PSD for snow, using a combined exponential and gamma distribution, but the simulation of polarimetric variables at and above the freezing level will be the focus of future studies. Table 1 summarizes whether a fixed value or predicted model variables are used to calculate and α for each hydrometeor category for each MP scheme.

Table 1.

Reference table for which predicted model variables are used to calculate the intercept (N0) and shape (α) parameters for rain (r), snow (s), graupel (g), and hail (h) from each of the microphysics schemes considered. The values are either fixed and listed, or calculated as a function of mixing ratio (q), number concentration (Nt), and/or air temperature (T). Schemes that do not contain a given hydrometeor category are listed as not applicable (NA).

Table 1.

d. Polarimetric radar observations

The upgraded WSR-88D radars provide domain-wide polarimetric observations that are used for comparison to the simulated variables. The Z and ZDR observations are filtered using a five-point along-the-radial median filter, while KDP is calculated from similarly filtered differential phase (ΦDP) observations using the least squares fit method of Ryzhkov and Zrnić (1996). Nine range gates are used when Z > 40 dBZ and 25 range gates are used for Z < 40 dBZ.

The availability of polarimetric observations allows for extensive quality control of the data using fuzzy logic (Park et al. 2009). The fuzzy logic method uses ranges of polarimetric radar data values and weights to determine the most likely hydrometeor type of the observation. The Z, ZDR, ρhv, standard deviation (SD) of Z (1-km running average), and SD(ΦDP) (2-km running average) membership functions are used along with their respective weights. The confidence vectors are not included. Additionally, the temperature profile and the presence of frozen hydrometeors from the forecast model are used to help further narrow down potential hydrometeor types before classification. For example, frozen categories are not considered at heights where full melting has occurred in the forecast and rain-associated categories are not considered above the freezing level. The MY forecast member was chosen since this scheme included the largest number of hydrometeor categories and produced storms with reasonable structure and intensity based on a qualitative comparison of the results. Those observations that are determined to be ground clutter, anomalous propagation, or biological scatterers are removed. This is important since SSEF forecasts begin at 0000 UTC and short-term forecasts in the late spring–early summer months will be at a time when observed radar blooms due to birds and insects are prominent (Lakshmanan et al. 2007). An example of Z, ZDR, and KDP observations at an elevation angle of 0.5° before and after the removal of nonmeteorological echoes for one of the cases evaluated in this study, at 0400 UTC 20 May 2013, is given in Fig. 2. The locations of the WSR-88D radars used are included as black dots in Fig. 2a. Obvious clutter from late evening radar blooms (2200 central standard time) is almost completely removed. Data points that are determined to be a three-body scatter spike are also removed (Mahale et al. 2014).

Fig. 2.
Fig. 2.

Mosaics of observed (a) reflectivity (dBZ), (b) differential reflectivity (dB), and (c) specific differential phase (° km−1) at a 0.5° tilt before ground clutter–biological scatterer removal and (d)–(f) after. The locations of the WSR-88D radars used are included as brown dots in (a).

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

3. Evaluation of simulation results

In this section, results for two example cases chosen from the 2013 Spring Experiment are evaluated. The first is a 4-h forecast initialized at 0000 UTC 20 May 2013 for a series of MCSs. The second is a 21-h forecast of several supercell thunderstorms also initialized at 0000 UTC 20 May 2013. These cases provide two different convective modes that contain distinct and different polarimetric signatures for evaluation. Additionally, the first case is a short-term forecast with sufficient lead time to allow for microphysical processes such as size sorting to develop while not too long for the storm systems originally initialized from radar data to dissipate. The divergence between the ensemble of model solutions and the observations is also relatively small at this point. Because of the early evening initialization time, a similar situation is difficult to find for supercell thunderstorms, which typically dissipate or grow upscale at night. For these reasons, the 21-h-long forecast valid in the afternoon of 20 May was chosen. It was also a point of emphasis to choose cases that had storm systems that were well placed so more focus could be put on the differences in the polarimetric variable values and not storm structure and placement. For convenience, the chosen ensemble members (section 2a) are referred to by their respective MP schemes: TOM (Thompson), MY (Milbrandt and Yau), MOR (Morrison), WDM (WDM6), and WSM (WSM6).

In this study, the forecast results are compared to the observations by creating a 0.5°-elevation mosaic of observed and simulated radar data from all WSR-88D sites within the domain from data in a “gridtilt” format for direct comparison. The simulated variables are left on the model grid in the horizontal but mapped (via weighted average) in the vertical to the elevation levels of each radar using the beam-pattern weighting function given in Xue et al. (2006). Conversely, the radar observations are interpolated onto the model grid points in the horizontal but left on the radar elevation levels in the vertical. As a result, both model and radar data are transferred to a common gridtilt space with respect to individual radars. A 0.5°-elevation mosaic is created by combining the lowest available elevation angles (0.5°) from the WSR-88D radars located within the domain, using the observation closest to the surface where two or more radars overlap. The 0.5°-elevation angle is chosen and the closest-to-the-surface value is used because polarimetric signatures associated with hydrometeor size sorting are most prevalent near the surface. Our method is similar to how the “reflectivity at lowest altitude” (RALA) product is produced in the Multi-Radar Multi-Sensor (MRMS) system (Smith et al. 2016). A near-surface constant-height mosaic will have large areas of missing observations because the radar beam height increases with distance. Additionally, vertical interpolation of radar elevation-level data to a constant model level can have large errors, as the radar beamwidth increases with distance from the radar (Sun and Crook 2001).

a. The 20 May 2013 mesoscale convective system case

Height falls associated with an upper-level trough moving into the central plains and ample low-level moisture led to the development of multiple areas of severe thunderstorms during the midafternoon of 19 May 2013. Over time, these clusters grew upscale to form several MCSs that stretched from the upper Mississippi valley south into Oklahoma. The most intense of these systems resulted from storms initially forming over central Kansas that continued into eastern Iowa. At 0400 UTC 20 May this system exhibited the elements of a classic MCS, including leading convection and trailing stratiform precipitation (Fritsch and Forbes 2001). Additional linear convective storms formed along an outflow-reinforced cold front that stretched southwestward into northern Oklahoma. Widespread damaging wind and hail was reported across the Midwest, and several tornadoes were reported in southwestern Missouri (SPC 2014a).

1) Qualitative evaluation of forecasts

Mosaics of observed and simulated Z, ZDR, and KDP in the gridtilt format described at the beginning of section 3 are plotted in Figs. 35. Locations of WSR-88D radar sites used for both the observed and simulated variable plots are included in Figs. 3a, 4a, and 5a. A 20-dBZ Z contour is included for reference in Figs. 4 and 5. Overall, the observed features are well placed in the five forecasts. However, the intensity and structure of the forecast precipitation differ from the observations, and differ among the forecast members. The MY and MOR members produce more widespread, high Z in the convective areas in eastern Iowa (Figs. 3c,d) compared to the observations but show a decrease in intensity in the stratiform precipitation region further west in central Iowa. In WDM and WSM, Z is lower than in MY and MOR in the convective areas and more closely matches the observations (Figs. 3e,f). However, the stratiform precipitation over Iowa is almost nonexistent in WDM, and is significantly underforecast in WSM. For the WDM case, Lim and Hong (2010) similarly found that WDM had low rain rates in the stratiform region of a 2D-simulated MCS and attributed these low rates to higher rain number concentrations and increased evaporation. The WSM and WDM forecasts are overall very similar as only warm rain processes are DM in WDM. The importance of predicting a second moment for ice processes was noted in Putnam et al. (2014), who found that a DM MY forecast, which predicts a second moment for snow and cloud ice, better maintained separate convective and stratiform precipitation regions in an MCS compared to an SM Lin forecast due in part to the improved transport of frozen hydrometeors between the convective towers and stratiform precipitation region. The TOM Z appears most reasonable in terms of both its intensity and coverage (Fig. 3b). The placement and structure of convection in western Missouri and northern Oklahoma, and the associated Z intensity, match the observations well. However, the convective region in eastern Iowa is disorganized, with no discrete linear convective line ahead of the trailing precipitation.

Fig. 3.
Fig. 3.

Mosaics of observed (a) reflectivity (dBZ) at a 0.5° tilt at 0400 UTC 20 May 2013 and simulated values at the same tilt from the (b) TOM, (c) MY, (d) MOR, (e) WDM, and (f) WSM forecasts. Locations of WSR-88D sites used for both the observed and simulated variable plots are noted with black dots in (a).

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

Fig. 4.
Fig. 4.

As in Fig. 3, but for differential reflectivity (dB). Features of interest referenced in the text are noted by arrowheads.

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

Fig. 5.
Fig. 5.

As in Fig. 3, but for specific differential phase (° km−1).

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

The range of Z may be due to different hydrometeors sizes, types, and water contents; simulated ZDR and KDP provide further insight to better differentiate the microphysical states of the members. Additionally, observed ZDR and KDP can be used to diagnose the hydrometeor categories using the fuzzy-logic method described in section 2d (Fig. 6). The observed hydrometeor classifications are computed for each radar and plotted as a 0.5°-elevation mosaic as described at the beginning of section 3. For the forecasts, the dominant category is determined based on which hydrometeor type provides the majority of the contribution to the linear simulated Z, including the diagnosed mixed-phase species described in section 2b. If no category contributes at least 50% to Z, the category is considered to be a “mix.” Since the model MP scheme categories do not match Park et al.’s (2009) HCA categories, hail, wet (melting) graupel, and mix are added to the classification list for identification when present in the PRDS results. Additionally, the forecast rain category does not differentiate between the “big drop” and “heavy rain” categories so all rain is combined into one category and wet (melting) hail from the PRDS results is considered “rain and hail.” Nonmeteorological categories are not included, as they were removed during the quality control process.

Fig. 6.
Fig. 6.

Mosaic of hydrometeor classification using fuzzy logic for (a) observations at a 0.5° tilt at 0400 UTC 20 May 2013 as well as classification of highest simulated linear reflectivity at the same tilt for the (b) TOM, (c) MY, (d) MOR, (e) WDM, and (f) WSM forecasts. Features of interest referenced in the text are noted by arrowheads. Locations of WSR-88D sites used for both the observed and simulated variable plots are noted with black dots in (a).

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

Observed ZDR is generally greater than 2.0 dB in the convective regions, with a maximum of around 3.0 dB, and is less than 2.0 dB in the stratiform regions (Fig. 4a). This matches the typical observed ZDR pattern in an MCS caused by differing rain PSDs; areas of convective precipitation have high ZDR because of the presence of large raindrops, with the maxima occurring along the leading edge of the convective line where the size sorting of smaller raindrops, which have a low terminal velocity and are carried farther rearward in the convective line, isolates larger raindrops and leads to high ZDR values. The stratiform region, which is not supported by an intense updraft, contains moderate precipitation with small- to medium-sized raindrops and lower ZDR values, respectively (Zhang et al. 2008). MY and MOR have the highest simulated ZDR results in convective regions, comparable to the observations, but the coverage of these high values is more widespread (Figs. 4c,d). The widespread high ZDR matches the areas of overforecast high Z, where the presence of larger raindrops in intense convection would be expected. The high ZDR, along with very high Z, is also indicative of large oblate wet graupel, the dominant hydrometeor category in these convective regions (Figs. 6c,d). The MY scheme predicts hail but shows a similar high bias in wet graupel as MOR. The MY scheme implemented in WRF-ARW as used in the CAPS SSEF was modified from Milbrandt and Yau (2005b) to include a strict minimum size threshold for hail. Hail below this threshold is converted back to graupel, and this threshold has been shown to result in forecasts that produce little, if any, hail (Van Weverberg et al. 2012). This may explain the presence of too much graupel compared to the observations in this case. There is an increase in ZDR values toward the leading edge of the convective lines in southeastern Kansas and northeastern Oklahoma, as well as in central Missouri and eastern Iowa (indicated by arrowheads in Figs. 4c,d); high ZDR at the leading edge of convection is a commonly found polarimetric signature associated with size sorting. However, there are many convective areas where large drops are embedded within the convection and a size-sorting signature is not evident. MOR has a strong size-sorting signature in southwestern Missouri (indicated by an A in Fig. 4d), but this corresponds with low Z and weak precipitation. It is not unusual to see high ZDR in developing convection as size sorting begins to occur, but the spike in ZDR may be because the drop breakup rate in MOR is dependent on qr, and the rate will be low where qr is small. As a result, in low-precipitation regions with small qr, the low rate of drop breakup may lead to a locally high number of large drops. These spikes in ZDR were also noted in Johnson et al. (2016).

There is a more significant difference in the microphysical state of MOR and MY in the stratiform precipitation region in central Iowa. There is widespread moderate ZDR in MOR, only 0.5 dB less than the ZDR in the convective line to the east, while ZDR decreases away from the leading convective lines in MY, similar to the observations. The distinguishing ZDR that differentiates the PSDs of the convective and stratiform regions is more prominent in MY; this matches the findings of Putnam et al. (2014). The difference in the similar DM MOR and MY forecast results highlights MP scheme challenges that extend beyond simply adding a second moment.

In TOM and WDM, ZDR shows no clear organization based on the structure of the convective systems (Figs. 4b,e). There is little difference between ZDR in the convective and stratiform regions, and no clear size-sorting signatures that match the observations. In fact, the highest ZDR in TOM is in an area of light precipitation on the rear side of the convective line in western Missouri, to the north and south of the convection in Iowa, and in isolated light showers in Missouri (indicated by arrowheads in Fig. 4b). Relatively high ZDR values are often seen with developing storms like those in Missouri, but the remaining noted ZDR patterns do not match the observations. Both TOM and WDM are DM for rain but only SM for graupel, and thus there is no size sorting of graupel, which has a greater impact on the development of low-level ZDR signatures (along with hail) than does the size sorting of rain (Dawson et al. 2014). Despite the apparent lack of size sorting, which is one of the features tied to maintaining a stratiform precipitation region in Putnam et al. (2014), TOM still represents the coverage of the convective and stratiform precipitation regions relatively well. Wheatley et al. (2014) found in their real-case EnKF study of an MCS that the Thompson scheme replicated the convective and stratiform regions well because of broad and intense development of snow aloft. Thompson is not DM for snow, but it does use a unique snow PSD and diagnostic N0s, and TOM contains stratiform coverage similar to that produced by the fully DM MOR and MY results. WDM also has areas where high ZDR is located along the rear edge of convection in northern Missouri and southern Iowa (indicated by arrowheads in Fig. 4e). In fact, ZDR is relatively low in the most intense areas of convection (Z > 40 dBZ), indicating a large number of small- to moderate-sized drops. Given that wet graupel is the dominant category present (indicated by arrowheads in Fig. 6e), it is likely that the wet graupel that exists is small and shedding small raindrops. This differs from MOR and MY, which also show a significant contribution to Z from wet graupel but have much higher ZDR values, indicating larger wet graupel and a significant difference in the graupel PSDs between the schemes.

Compared to Z, ZDR differs more substantially between WDM and WSM (Figs. 3e,f and 4e,f). The ZDR maxima from the two schemes are about the same, but in WSM Z and ZDR have a monotonic relationship, with the highest ZDR collocated with the highest Z; this is because the WSM scheme with a fixed N0 is incapable of size sorting. WDM does not improve upon WSM, however, compared to the observations, since Z and ZDR do not differ between convective and stratiform precipitation regions and the size-sorting signatures appear in the wrong locations. WDM is DM for rain and cloud droplets, but SM for ice species as in WSM, further emphasizing the impact that the size sorting of graupel has on low-level rain PSDs and the associated ZDR signatures.

Simulated KDP is generally lower in all members compared to the observations with values mostly less than 2.0° km−1. A few more intense convective areas (Z > 50 dBZ) in MY and WSM have KDP higher than 2.0° km−1 (Figs. 5c,f) which agrees more closely with the observations (Fig. 5a). MY and MOR have similar ZDR maxima but generally lower KDP values compared to the observations, so the rain PSDs must contain a lower concentration of small- to moderate-sized drops (Figs. 5a,c,d). The convective regions in MY, MOR, and WDM contain significant wet graupel (Figs. 6c–e). Low ZDR and KDP are indicative of small wet graupel in WDM (Figs. 4e and 5e) while significantly higher ZDR in MY and MOR (Figs. 4c,d) suggests larger wet graupel but with a low water ratio. Low ZDR is also indicative of the lack of large raindrops in WDM due to the gamma distribution with a short tail in the large drop’s end. WSM has lower ZDR but similar KDP (Fig. 5f) compared to MY and similar ZDR but higher KDP compared to TOM, indicating a higher concentration of small- to moderate-sized drops due to a large fixed N0r. A high bias in graupel compared to the observations (Fig. 6a) could potentially explain the low KDP (as compared to an all-rain scenario), but WSM, as well as TOM, have similarly low KDP values in areas of pure rain.

Another notable difference in the forecasts compared to the observations is the lack of significant KDP in the stratiform precipitation region over central Iowa. Those members that show this trailing precipitation (TOM, MY, and MOR) only have noteworthy KDP where Z exceeds 35 dBZ (Figs. 5b–d); the observations exhibit low (but consistent) KDP throughout the stratiform region (Fig. 5a). Although the melting layer differs between the forecasts, which could contaminate the KDP results, KDP is also lower in areas of pure rain. In MOR, ZDR is higher than observed in the stratiform region, indicating PSDs with a few larger raindrops and lower water content overall (Fig. 4d). In TOM and MY, ZDR is closer to the observations but similarly low KDP values suggest an overall low bias among the DM schemes in the concentration of moderately sized raindrops and their water content (Figs. 5b,c).

2) Quantitative evaluation of forecasts

Although forecast errors at all scales continuously grow as forecast length increases, the error growth rate is much larger for smaller-scale phenomenon (e.g., convective cells). Additionally, diagnostic metrics are highly sensitive to displacement errors for storm-scale forecasts, especially for finescale structures, making one-to-one quantitative verification even more difficult. A few different methods of quantitative comparison that help account for these errors are considered.

Percentile histograms of simulated Z, ZDR, and KDP are calculated over the domain used for Figs. 35 to gauge the overall distribution of values (Fig. 7). The histograms are created by first ranking all observed and simulated values individually and then distributing the simulated values within 10 bins representing the observed percentiles between 0.0 and 1.0. Percentiles relative to the observations are used to account for potential biases and outlier values in the model results. The range of the observed values corresponding to the 0.0–0.2, 0.2–0.4, etc. percentile bins is indicated by coordinating the colors between the observed values used as the bounds for their respective percentile bins. For reference, the observed histograms for each variable are provided in Figs. 7a–c, with a black line indicating the number of observed values per bin included for all simulated results and a red line indicating the ideal distribution of the simulated model values, if these values were distributed evenly. The total number of simulated values is also included in each subplot.

Fig. 7.
Fig. 7.

Histograms of observed (a) reflectivity (dBZ), (b) differential reflectivity (dB), and (c) specific differential phase (° km−1) mosaic values at a 0.5° tilt from Figs. 35, as well as percentile histograms of simulated values at the same tilt from the (d)–(f) TOM, (g)–(i) MY, (j)–(l) MOR, (m)–(o) WDM, and (p)–(r) WSM forecasts distributed into bins based on the observed percentiles (noted by the horizontal solid black line). The idealized distribution given the number of values in each of the simulated results is indicated by the horizontal red line. The total number of values is noted in the top-right-hand corner of each panel. The observed values are used as the bounds for the percentile bins and are similarly color coded in both the observed and simulated value plots.

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

There is a high bias in the simulated Z results relative to the largest observed Z values, indicative of strong convective precipitation, especially for MY and MOR (Figs. 7g,j). Qualitative comparisons suggested that the convective regions contained more intense precipitation over a larger area and that wet graupel contributed to high Z. The observed distribution peaks at around 35 dBZ (Fig. 7a), which corresponds to Z from the large region of stratiform precipitation in central Iowa and contributes to the number of values in the middle percentiles. The MOR and TOM distributions are most similar to the observations in this midrange, though all forecasts have at least some low bias. MOR has a consistent high bias for low and high Z because of an overforecast of precipitation coverage (Fig. 7j). Conversely, WDM has a consistent low bias at nearly all percentiles, lacking the stratiform precipitation and significantly underforecasting precipitation coverage overall (Fig. 7m).

The observed distribution of ZDR peaks around 1 dB and extends up to 4 dB (Fig. 7b). TOM, and particularly MY and MOR, have high biases in the higher percentiles because of the overforecast east–west extent of the convective lines as well as PSDs that contain too many large drops. MOR has a significant bias above the 90th percentile (Fig. 7k) related to the large drops in the stratiform region, oblate wet graupel in the convective region, and the overforecasting of precipitation coverage overall. On the other hand, WDM and WSM have similar low biases above the 50th percentile. The low bias in WSM may be related to the relatively high fixed N0r of 8 × 106 m−4 (Fig. 7q). Although WDM is DM for rain, its behavior is similar to the SM WSM, which was also noted in the qualitative evaluation.

The KDP histograms are limited to values above 0.5 ° km−1 because values below this threshold can be indistinguishable from noise in the observations (Jung et al. 2008b). This limits the KDP assessment to the convective regions, where the precipitation coverage for all members matches the results better as well. The TOM, WDM, and WSM results are relatively similar to the observations for the low to middle percentiles while MY and MOR overestimate these values because of the broader width of the convective lines in these forecasts (Figs. 7f,i,l,o,r). However, for the highest percentiles, which represent the peak of the observed KDP values in the intense precipitation convective regions, there is a low bias in all members. In general, all members have simulated KDP values lower than observed in the convection regions as a result of apparent lower liquid water contents resulting from contamination by graupel (MY, MOR, and WDM; see Figs. 7i,l,o), a bias toward larger drops as indicated by ZDR (TOM, MY, and MOR; see Figs. 7f,i,l), and a high concentration of small drops due to the fixed intercept parameter in WSM (Fig. 7r).

Another quantitative measure for evaluating the PSDs is scatterplots of Z versus ZDR at a given location for the observations and forecasts (Fig. 8). Data points where Z > 5 dBZ from the Figs. 35 domain are considered and are categorized by dominant hydrometeor type using the same process as for Fig. 6. TOM, MY, and MOR show a broad overall distribution of Z and ZDR value combinations similar to the observations. The additional free PSD parameter in the DM scheme allows for greater flexibilities in the range of possible PSDs in the forecast. The high density of data points for rain in the observations results from the broad region of stratiform precipitation (Fig. 8a). TOM, MOR, and MY, which performed well in terms of stratiform precipitation coverage in the qualitative evaluation, have a similar concentration of data points for rain (Figs. 8b–d). The distribution in MOR is shifted toward slightly higher Z and ZDR because of the widespread coverage of more moderate to large raindrops in the stratiform region. The large amount of melting ice species (snow, graupel, or a mix of both; see Fig. 8d) leads to the overforecast convective intensity in MOR based on a comparison of the hydrometeor types associated with these values in Figs. 3d and 6d; the same is true to a lesser extent with MY (Fig. 8c). The ZDR maxima displaced from the leading edge of the convection in WDM result from wet snow and graupel (Fig. 8e, indicated by arrowheads in Figs. 4e and 6e), while that with TOM is mainly associated with rain (Fig. 8b, indicated by arrowheads in Figs. 4b and 6b).

Fig. 8.
Fig. 8.

Scatterplot of (a) observed reflectivity (dBZ) and differential reflectivity (dB) mosaic values at a 0.5° tilt from Figs. 35, as well as scatterplots of simulated values at the same tilt from the (b) TOM, (c) MY, (d) MOR, (e) WDM, and (f) WSM forecasts.

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

WDM and WSM exhibit very similar distributions (Figs. 8e,f). There is little spread in the data points in WSM given the one-to-one relationship between Z and ZDR in an SM scheme. Again, WDM, being only DM for rain, and with a diagnostic N0s, exhibits the least variation compared to the more complex TOM and fully DM MOR and MY. Most of the variation in both WSM and WDM is associated with the presence of mixed-phase precipitation where changing liquid and frozen water contents lead to various Z and ZDR combinations.

Since traditional numerical measures like root-mean-square error (RMSE) will indicate poor results when spatial errors are present, neighborhood methods have been developed to account for placement errors when the overall storm structure is otherwise good (Ebert 2008). One of these techniques, the fractions skill score (FSS; Roberts 2008; Roberts and Lean 2008), has been considered in past studies involving the CAPS SSEF and is used again here (Schwartz et al. 2009; Cintineo et al. 2014). The FSS is calculated by finding the fraction of forecast grid points in a neighborhood with a given radius that exceeds a threshold value compared to the observations. The FSS is designed so that as the radius for the neighborhood increases to the size of the domain, the score will asymptote toward an ideal finite value of 1. If there is bias present in the forecast, then the score will be less than 1, except for relatively small-scale neighborhoods (Mittermaier and Roberts 2010). A forecast can be considered to have measurable skill when
e2
where is the domain-wide fraction of grid points where observations exceed the given threshold (Roberts and Lean 2008).

The FSS is calculated over the Figs. 35 domain for several thresholds for radii ranging from 0 to 200 km to account for the regional nature of the MCS coverage (Fig. 9). Overall precipitation coverage, including both the convective and stratiform regions, is assessed using a threshold of Z > 15 dBZ (Fig. 9a). All forecasts perform well at this threshold. TOM, MY, and MOR show skill, with initial scores around 0.65 that increase to about 0.9 at the 100-km radius. WDM has the worst scores overall, averaging around 0.2 less than other members, likely because of the low-precipitation coverage bias. When the Z threshold is increased to 40 dBZ, to assess the prediction of intense convective precipitation (Fig. 9b), there is more spread between the members, and all members exhibit very poor scores for small radii when large spatial errors are present. Interestingly, WDM has the highest score for the 100-km radius because of the less extensive east–west coverage bias of the convective lines compared to the other members. WDM underforecasts precipitation overall but matches the observations better in the convective regions. TOM and MY show some skill using a 100-km radius, but do not improve much with increased radii. MOR, which exhibits substantial high Z bias due to the overforecast of intense convective precipitation, has no measurable skill.

Fig. 9.
Fig. 9.

FSSs for the TOM, MY, MOR, WDM, and WSM forecast results at increasing neighborhood radii for (a) reflectivity values > 15 dBZ, (b) reflectivity values > 40 dBZ, (c) differential reflectivity values > 2.5 dB, and (d) specific differential phase values > 0.6° km−1 for the mosaics in Figs. 35. The horizontal black line indicates skill greater than a random forecast.

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

A ZDR threshold of 2.5 dB is chosen to assess convective regions in the observations where the largest drops are present, specifically the maxima seen with the polarimetric signatures associated with size sorting on the leading edges of convective lines (Fig. 9c). Scores are generally poor; the only skillful forecasts are TOM, which shows skill at the 100-km radius, and MY, which shows skill at the 150-km radius. This result indicates the coverage and intensity of significant ZDR in TOM in convective regions is closest to the observations without overforecasting large drops overall within the stratiform region. However, the qualitative evaluation in section 3a(1) showed ZDR in TOM is displaced and large radii neighborhoods miss these finescale details. MOR is likely negatively impacted by the high ZDR coverage bias in the stratiform region and greater east–west extent of the convective regions. WDM and WSM are biased toward small hydrometeor sizes.

Similarly to ZDR, a threshold of 0.6° km−1 for KDP is chosen to highlight the convective cores where higher liquid water content is present (Fig. 9d). The KDP maximum is generally lower in all members than in the observations and skill scores at higher thresholds will be very poor. All members have skill for radii greater than 50 km (Fig. 9d). Thus, convective regions with high KDP are relatively well placed, with the caveat that graupel contamination may affect the upper range of these values. The better FSS scores for KDP compared to ZDR are likely due to the more direct linkage between high liquid water content and intense convection while ZDR patterns associated with size sorting are not collocated with Z maxima. The quantity ZDR provides a more stringent assessment of microphysical processes and states.

Since the range of simulated polarimetric variable values in each forecast may not match the overall range of the observations, the FSS scores are calculated again for the same thresholds using percentiles (Fig. 10). The percentile value in the observations consistent with each numeric threshold is used as the threshold to assess the forecast percentile values. This method normalizes the scores for those forecasts that do not produce values as high as the observations, effectively removing biases and providing a fairer assessment of feature placement, as in the ZDR values for WDM and WSM. The scores for Z > 15 dBZ and KDP > 0.6° km−1 are very similar to the previous results. However, the new scores are improved significantly for the Z > 40 dBZ convective assessment and ZDR > 2.5 dB, which are more affected by maximum value biases. More specifically, those forecast members that showed no skill without using percentiles (MOR for Z > 40 dB; TOM, WDM, and WSM for ZDR > 2.5 dB) now show skill at higher radii when using percentiles. The most significant improvement is in WDM and WSM; these members have a low bias compared to the other members for high ZDR values. The percentile calculations show the highest ZDR values from WDM and WSM forecasts are well placed compared to the observations but underestimated in value. Future quantitative assessment methods of polarimetric variables may need to take into account maximum and minimum value biases.

Fig. 10.
Fig. 10.

As in Fig. 9, but with the FSS scores calculated based on percentile values relative to the observations.

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

b. The 20 May 2013 supercell case

Several supercell thunderstorms developed along a stationary front across the southern plains during the early afternoon of 20 May 2013, the most intense of which occurred over central and southern Oklahoma. Dewpoints in the low 70s °F and 5000 J kg−1 of CAPE combined with winds in excess of 50 kt (where 1 kt = 0.51 m s−1) at 500 hPa associated with an upper-level trough to create a volatile severe weather environment. The most intense storm, which produced a tornado rated EF-5 on the enhanced Fujita scale (EF scale), formed along the stationary front southwest of the Oklahoma City, Oklahoma, area shortly before 2000 UTC. The tornado killed 24 people and caused over 1 billion dollars in damage across the southern Oklahoma City metropolitan area (NWS 2014). Additional tornadoes were reported across Oklahoma along with widespread large hail reports over southern Oklahoma (SPC 2014b).

1) Qualitative evaluation of forecasts

Mosaics of 0.5°-tilt observed and simulated Z, ZDR, and KDP for all members at 2100 UTC (Figs. 1113), as well as hydrometeor classifications using the same process in section 3a(1) (Fig. 14), are evaluated. Locations of WSR-88D radar sites within the domain used to create the observed and simulated variable mosaics are included in Figs. 11a, 12a, 13a, and 14a. A 20-dBZ Z contour is included for reference in Figs. 12 and 13. The placement and the coverage of the forecast convection are worse than in the MCS case because of the longer forecast lead time, the isolated nature of discrete supercell storms, and the fact that the storm development is not directly influenced by assimilated radar data at the IC time. The Z patterns for the southern Oklahoma storms in MY and MOR (indicated by an A in Figs. 11c,d) both exhibit classic supercell structure with a hook-echo–rear-flank downdraft, indicative of the presence of a mesocyclone. It should be noted that the structures are rather large compared to the observations; this is often seen in forecasts using 4-km grid spacing (Lean et al. 2008; Johnson et al. 2013). TOM, WDM, and WSM have a line of cells that are smaller and have low precipitation in comparison but have supercell characteristics (Figs. 11b,e,f). For reference, a plot of the Spring Experiment hourly maximum updraft helicity product (Kain et al. 2008; Kong 2013) for 2100 UTC is included (Fig. 15); the high values greater than 150 m2 s−2 for the southern Oklahoma storm in all members are indicative of a mesocyclone (indicated by arrowheads in Fig. 15). As in the MCS case, WDM underforecasts the precipitation coverage, while MOR forecasts the most widespread, high Z. In the observations (Fig. 11a), Z generally peaks at a higher value (Z > 50 dBZ) than in the model forecasts; MOR is most similar to the observations. However, the high Z in MOR is due to the presence of wet graupel (Fig. 14d), while high Z observations are mostly due to rain and some hail. There is spurious convection in the northwest corner of the domain in all members.

Fig. 11.
Fig. 11.

Mosaics of observed (a) reflectivity (dBZ) at a 0.5° tilt at 2100 UTC 20 May 2013 and simulated values at the same tilt for the (b) TOM, (c) MY, (d) MOR, (e) WDM, and (f) WSM forecasts. Features of interest referenced in the text are noted by capital letters. Locations of WSR-88D sites used for both the observed and simulated variable plots are noted with black dots in (a).

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

Fig. 12.
Fig. 12.

As in Fig. 11, but for ZDR (dB). Features of interest referenced in the text are noted by arrowheads and numbers.

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

Fig. 13.
Fig. 13.

As in Fig. 11, but for specific differential phase (° km−1).

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

Fig. 14.
Fig. 14.

Mosaic of hydrometeor classification using fuzzy logic for (a) observations at a 0.5° tilt at 2100 UTC 20 May 2013, as well as classification of the highest simulated linear reflectivity at the same tilt for the (b) TOM, (c) MY, (d) MOR, (e) WDM, and (f) WSM forecasts. Features of interest referenced in the text are noted by arrowheads. Locations of WSR-88D sites used for both the observed and simulated variable plots are noted with black dots in (a).

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

Fig. 15.
Fig. 15.

2D plots of maximum updraft helicity (m2 s−2) at 2100 UTC 20 May 2013 for the (a) TOM, (b) MY, (c) MOR, (d) WDM, and (e) WSM forecasts. Features of interest referenced in the text are noted by arrowheads.

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

The observed ZDR is generally higher on the right (southeastern) edge of the forward flanks of the observed cells (Fig. 12a), exhibiting a distinctive ZDR arc. In the center of the forward flanks of the central Oklahoma storms, ZDR is lower, possibly because of a hail-induced ZDR hole, but the HCA does not identify widespread, consistent areas of hail (Fig. 14a). The lower ZDR is associated with higher KDP (Fig. 13a), indicating moderately sized drops and a high rain rate. Along the right-forward flank of the dominant southern Oklahoma storm in MOR, MY, and to a lesser extent in WDM ZDR increases (indicated by arrowheads in Figs. 12c–e), while other convective cells are less organized and do not show this signature. The HCA identifies wet graupel in MOR, MY, and WDM along the right-forward flank of this storm (indicated by arrowheads in Figs. 14c–e), and the size sorting of melting graupel has been shown by Dawson et al. (2014) to have a substantial impact on the model representation of the ZDR arc. MOR shows the highest ZDR farther downwind of the forward flank than the extent of the wet graupel, consistent with Dawson et al. (2014; see their Fig. 17). Wet graupel is present along the entire right-forward flank in MY and WDM, and the maximum ZDR is not located along the immediate edge. Dawson et al. (2014) also found hail better replicated the observed coverage and intensity of the ZDR arc compared to graupel, which can lead to an overextensive forward flank, as seen in this case.

TOM and WSM do not exhibit the same ZDR pattern as the observations (Figs. 12b,f). There is a one-to-one relationship in WSM where high ZDR occurs with high Z in the center of the cells. Like the MCS case, a relative ZDR maximum occurs along the edges of smaller cells with less intense precipitation in TOM, MOR, and WDM. This occurs in the southwest portion of the domain for MOR and TOM, in northwest Oklahoma for MOR, and in central Oklahoma for TOM and WDM (indicated by 1, 2, and 3 in Figs. 12b,d,e, respectively). As noted for the MCS case, high ZDR is sometimes associated with aggressive size sorting in developing convection. The drop breakup scheme in MOR may have also contributed to the spikes in ZDR values associated with weak precipitation in that case. The ZDR in TOM appears to be particularly misplaced; there is a large area of ZDR exceeding 2.5 dB that occurs between the areas of more intense precipitation associated with the storm cells. The PSDs in this region are heavily weighted toward a few large drops given the low Z. Compared to MY and MOR, TOM, WSM, and WDM are not DM for graupel, which was noted to have an impact on size sorting in supercells.

In all members, KDP is underforecast compared to the observations. The observations peak above 3.0° km−1 (Fig. 13a), while only TOM has a maximum above 1.75° km−1 (Fig. 13b). Of note, the highest KDP values are not collocated with the highest ZDR values in the southern Oklahoma storm in MY and MOR (Figs. 13c,d). In the center of the storm KDP has a relative maximum but decreases in the right-forward flank, where ZDR is higher; another indication that size sorting has resulted in a few large raindrops in the ZDR arc compared to elsewhere in the forward flank. Although there is some graupel present in the forecast that may contaminate the KDP results, particularly in MOR, most of the forecast convection is classified as pure rain, suggesting liquid water content is underforecast overall.

2) Quantitative evaluation of forecasts

As in section 3a(2), percentile histograms (Fig. 16) and FSSs (Fig. 17) are considered for quantitative evaluation. Both are calculated over a subdomain that focuses on the line of supercells that extends from southeast Kansas to northwest Texas. The MY and MOR histograms show an overforecast of precipitation coverage (Figs. 16g,j); this overforecast is present but confined to lower percentiles in TOM and WSM (Figs. 16d,p). Given this circumstance, the mostly even distribution of intensities in the observations (Fig. 16a) is matched relatively well by these members. The WDM precipitation coverage is underforecast so substantially that the Z distribution is lower than the observations for all percentiles (Fig. 16m).

Fig. 16.
Fig. 16.

As in Fig. 7, but from Figs. 1113.

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

Fig. 17.
Fig. 17.

FSSs for the TOM, MY, MOR, WDM, and WSM forecast results at increasing neighborhood radii for (a) reflectivity values > 15 dBZ, (b) differential reflectivity values > 2.5 dB, and (c) specific differential phase values > 0.4° km−1 for the mosaics in Figs. 1113. The black line indicates skill greater than a random forecast.

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

There is a high bias in the amount of ZDR values because of the greater precipitation coverage in TOM, MY, and MOR. In MY, ZDR has a relatively even distribution compared to the observations, with the caveat that the overforecast coverage of precipitation leads to a consistently higher number of values at most percentiles overall (Fig. 16h). After considering the grid scale, simulated ZDR in MY represents the varying degree of maximum raindrop size in the rain PSDs well. MOR has a significant peak at the 70th percentile (Fig. 16k), corresponding to the widespread ZDR around 2.5 dB and greater in the central forward-flank regions where wet graupel is present (Fig. 14d). In WSM, for all but the lowest percentiles, ZDR is lower than the observations compared to TOM, MOR, and MY (Fig. 16q); this is expected given the fixed intercept parameters used in this SM scheme. In WDM, ZDR has a significant low bias (Fig. 16n) because of small raindrops and small, wet graupel in precipitation underforecast in coverage and intensity.

In the forecast KDP distributions (Figs. 16f,i,l,o,r), all members but WDM have a similar number of values, with a generally decreasing trend overall toward the higher percentiles. WDM has the lowest number of values compared to the observations while WSM has the highest number of values in comparison, particularly for the lowest percentiles. All forecast members appear to have generally lower liquid water contents than the observations in pure rain areas.

The FSS is calculated for radii only up to 100 km as a result of the more localized nature of the supercell case. All members show some skill for radii of 20–40 km or more for a Z threshold of 15 dBZ (Fig. 17a). WSM and TOM have noticeably higher scores than the other members because the precipitation coverage is more similar to the observations; precipitation coverage is overforecast in MY and MOR and underforecast in WDM. The qualitative evaluation showed more realistic Z and ZDR patterns in MY and MOR compared to WSM and TOM but the size of the supercells in the former cases was notably larger. Members generally show no skill for ZDR (Fig. 17b). The KDP threshold (Fig. 17c) is decreased slightly compared to the MCS case since the simulated values are lower overall (0.4° km−1 instead of 0.6° km−1). MY and WSM, as in the MCS case, have the best skill for KDP for large radii (>60 km), but these scores are not high (<0.7). Issues related to grid scale and storm placement leave many quantitative challenges for the simulated polarimetric variables, especially in terms of ZDR patterns for a supercell case.

Figure 18 replicates the FSS calculations for Fig. 17 using percentile values. As in the MCS case, those forecasts with very poor scores for ZDR > 2.5 dB due to lower maximum values than the observations (WDM and WSM) show significant improvement. However, all forecasts still show little to no skill overall. The spatial extent and coverage of ZDR signatures for the supercell case appear more difficult to match than for the MCS case. Unlike the MCS case, the KDP scores are also improved for all members. The histograms for each case show that the distribution of KDP values is a better match in the MCS case than for the supercell case where there is a greater low bias in KDP values. The use of percentiles helps better match the observed and forecast distributions for comparison so that all but WDM show at least some skill at the larger radii values. It is clear the poor KDP scores when percentiles are not used are due to the low bias of values in the forecast.

Fig. 18.
Fig. 18.

As in Fig. 17, but with the FSS scores calculated based on percentile values relative to the observations.

Citation: Monthly Weather Review 145, 1; 10.1175/MWR-D-15-0415.1

4. Summary and conclusions

Polarimetric variables are simulated from the CAPS Spring Experiment Storm-Scale Ensemble Forecasts (SSEFs) for evaluation of both single-moment (SM) and double-moment (DM) model microphysics (MP) schemes. An existing polarimetric radar data simulator (PRDS; Jung et al. 2008a; Jung et al. 2010) is modified to add several new MP schemes including Thompson (TOM), Morrison (MOR), and WDM6 (WDM); Milbrandt and Yau (MY) and WSM6 (WSM) were already included. Careful attention is paid in the simulation to the hydrometer types and particle size distributions (PSDs) of each scheme to properly represent the forecast microphysical state. Two cases are considered: a 4-h forecast for a series of mesoscale convective systems (MCSs) from 20 May 2013 and a 21-h forecast of supercell thunderstorms from the 20 May 2013 Oklahoma tornado outbreak. Simulated reflectivity (Z), differential reflectivity (ZDR), and specific differential phase (KDP) from a single ensemble member forecast using each scheme with otherwise similar model settings are compared to observations from the recently upgraded WSR-88D radar network.

In MOR and MY in the supercell case, ZDR, as well as classification of the hydrometeors present, produce results consistent with Dawson et al. (2014), who demonstrated the role that the size sorting of graupel plays in the formation of the ZDR arc. The other schemes examined are not DM for graupel and do not show this pattern. In addition, the two schemes that best represent polarimetric size-sorting signatures (MY and MOR) also show better coverage of stratiform precipitation compared to the SM WSM scheme. TOM, only DM for rain with a unique snow PSD and diagnostic N0s, shows incorrect size-sorting signatures but still represents the stratiform precipitation region well. Qualitative and quantitative evaluation shows that WDM, despite being DM for rain, has a similar one-to-one relationship between Z and ZDR as WSM and no stratiform precipitation development. The other DM schemes include more complex diagnostic equations (TOM) or are fully DM (MY and MOR), demonstrating that size sorting of hydrometeor categories in addition to rain is as important in improving the forecast microphysical state. TOM, MOR, and WDM all have incorrect ZDR maxima associated with isolated, weak convection on the back side of convective lines where isolated large drops are not expected.

Notable biases are present in each scheme. Both Z and ZDR in the stratiform precipitation region of the MCS are too high in TOM, MY, and particularly MOR, indicating that the forecast rain PSDs contain too many large drops for stratiform rain. The MY, MOR, and WDM forecasts contain a large amount of wet (melting) graupel in convective areas, as determined by the coexistence of rain and graupel in the model, while a hydrometeor classification algorithm (HCA) used indicates a small amount of hail but mostly rain in similar locations in the observations. Although wet graupel is not included as a category in the classification scheme, significant graupel would not be expected near the surface for these warm season cases. These areas of wet graupel contribute to more extensive intense Z compared to the observations. MY includes a hail category but contains a similarly significant amount of graupel, likely due to a strict minimum hail size threshold in the scheme. Finally, simulated KDP values are lower in all members for both cases, particularly in intense convective precipitation regions. We find that KDP increases with large amounts of moderate-sized drops and higher liquid water contents, but large raindrops and graupel with a low water ratio are apparent in TOM, MOR, and MY, while WDM and WSM have a bias toward small raindrops and graupel. The use of a triple-moment (TM) MP scheme with an effectively variable shape parameter would provide greater flexibility to represent a wider range of possible PSDs, including those that have a positive shape parameter ) with a brace-like shape. This leads to a maximum of moderately sized drops and higher liquid water contents–rain rates compared to an exponential distribution, which tends to underestimate the liquid water content. Additionally, KDP is a measurement related to mass in a volume. With a 4-km grid the volumes are quite large for calculating KDP, which may vary greatly over a few kilometers distance, and more localized maxima that may be present could be missed, particularly in intense convective precipitation areas.

There are several challenges inherent in large-domain storm-scale forecasts that can hamper our ability to gain information about the different MP schemes from the simulated variables. A poor forecast of storm structure for a given supercell or MCS will be missing notable polarimetric value patterns. For example, TOM, WDM, and WSM have poor supercell structures that make ZDR arc comparisons more difficult. Previous studies have shown that forecasts performed using a 4-km horizontal grid spacing may miss some finescale details in convection (Bryan et al. 2003), result in larger-scale structures (Lean et al. 2008; Johnson et al. 2013), and impede processes such as the development of trailing stratiform precipitation (Bryan and Morrison 2012; Xue et al. 2013). Other studies that have considered simulated polarimetric variables use a smaller grid scale than 4 km: 2 km in Putnam et al. (2014), 1 km in Jung et al. (2012), and 1 km in Li and Mecikalski (2012). These patterns may also be displaced compared to observations, making quantitative comparisons difficult.

Forecast members show some skill in terms of the fractions skill score (FSS) for Z and KDP in the MCS case but higher scores require larger radii, and all forecasts exhibit very poor skill for ZDR in both cases. Although qualitative comparisons indicate that MY and MOR represent ZDR patterns relatively well, substantial spatial error leads to FSS scores with no skill. Normalizing the FSS using percentile values results in a significant improvement in skill for forecasts that do not contain simulated values as high as the observations. Future studies should continue to adapt these methods as forecasts are refined and improved before general statistics can be produced for all forecasts over the Spring Experiment period. Such information could be used in the future to provide additional forecast products as well as serve research purposes like determining which MP scheme may best represent polarimetric signatures in supercells for use in dual-polarization data assimilation experiments.

Finally, we point out that there are also many uncertainties with the polarimetric radar simulator. There are various assumptions made on the water drop aspect ratio; canting angle of snow, hail, and graupel; and water fraction for mixed-phase species. These are some of the aspects that still need refinement and tuning, and they can affect the microphysics evaluation. Dawson et al. (2014) developed an alternative water fraction model for the mixed phases that depends on the size spectrum. The relative performance of this model should be evaluated in the future.

Acknowledgments

This research was primarily supported by NSF Grant AGS-1046171. BJP and GZ were also supported by NOAA Grant NA11OAR4320072. MX and YJ were also supported by NSF Grant AGS-1261776. The CAPS Spring Experiment ensemble forecasts and FK were supported by NOAA Collaborative Science and Technology Applied Research (CSTAR) Grant NA13NWS4680001, and the forecasts were produced on a supercomputer operated by NICS at the University of Tennessee as part of the NSF XSEDE resources. A research grant from the Korea Meteorological Administration supported improvements to the polarimetric radar data simulator. Detailed comments from Dr. Edward Mansell and two anonymous reviewers significantly improved the original manuscript.

REFERENCES

  • Brandes, E. A., G. Zhang, and J. Vivekanandan, 2002: Experiments in rainfall estimation with a polarimetric radar in a subtropical environment. J. Appl. Meteor., 41, 674685, doi:10.1175/1520-0450(2002)041<0674:EIREWA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Bringi, V. N., and V. Chandrasekar, 2001: Polarimetric Doppler Weather Radar. Cambridge University Press, 636 pp.

  • Bryan, G. H., and H. Morrison, 2012: Sensitivity of a simulated squall line to horizontal resolution and parameterization of microphysics. Mon. Wea. Rev., 140, 202225, doi:10.1175/MWR-D-11-00046.1.

    • Search Google Scholar
    • Export Citation
  • Bryan, G. H., J. C. Wyngaard, and J. M. Fritsch, 2003: Resolution requirements for the simulation of deep moist convection. Mon. Wea. Rev., 131, 23942416, doi:10.1175/1520-0493(2003)131<2394:RRFTSO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Chen, F., and J. Dudhia, 2001: Coupling an advanced land surface–hydrology model with the Penn State–NCAR MM5 modeling system. Part I: Model implementation and sensitivity. Mon. Wea. Rev., 129, 569585, doi:10.1175/1520-0493(2001)129<0569:CAALSH>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Cintineo, R., J. A. Otkin, M. Xue, and F. Kong, 2014: Evaluating the performance of planetary boundary layer and cloud microphysical parameterization schemes in convection-permitting ensemble forecasts using synthetic GOES-13 satellite observations. Mon. Wea. Rev., 142, 163182, doi:10.1175/MWR-D-13-00143.1.

    • Search Google Scholar
    • Export Citation
  • Clark, A. J., and Coauthors, 2012: An overview of the 2010 Hazardous Weather Testbed Experimental Forecast Program Spring Experiment. Bull. Amer. Meteor. Soc., 93, 5574, doi:10.1175/BAMS-D-11-00040.1.

    • Search Google Scholar
    • Export Citation
  • Dawson, D. T., II, E. R. Mansell, Y. Jung, L. J. Wicker, M. R. Kumjian, and M. Xue, 2014: Low-level ZDR signatures in supercell forward flanks: The role of size sorting and melting of hail. J. Atmos. Sci., 71, 276299, doi:10.1175/JAS-D-13-0118.1.

    • Search Google Scholar
    • Export Citation
  • Du, J., J. McQueen, G. DiMego, Z. Toth, D. Jovic, B. Zhou, and H. Chuang, 2006: New dimension of NCEP Short-Range Ensemble Forecasting (SREF) system: Inclusion of WRF members. Preprints, Expert Team Meeting on Ensemble Prediction System, Exeter, United Kingdom, WMO. [Available online at https://www.wmo.int/pages/prog/www/DPFS/Meetings/ET-EPS_Exeter2006/Doc6(5).pdf.]

  • Ebert, E. E., 2008: Fuzzy verification of high-resolution gridded forecasts: A review and proposed framework. Meteor. Appl., 15, 5164, doi:10.1002/met.25.

    • Search Google Scholar
    • Export Citation
  • Fritsch, J. M., and G. S. Forbes, 2001: Mesoscale convective systems. Severe Convective Storms, Meteor. Monogr., No. 50, Amer. Meteor. Soc., 323–357.

  • Gao, J.-D., M. Xue, K. Brewster, and K. K. Droegemeier, 2004: A three-dimensional variational data analysis method with recursive filter for Doppler radars. J. Atmos. Oceanic Technol., 21, 457469, doi:10.1175/1520-0426(2004)021<0457:ATVDAM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hodur, R. M., 1997: The Naval Research Laboratory’s Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS). Mon. Wea. Rev., 125, 14141430, doi:10.1175/1520-0493(1997)125<1414:TNRLSC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hong, S.-Y., and J.-O. J. Lim, 2006: The WRF single-moment 6-class microphysics scheme (WSM6). J. Korean Meteor. Soc., 42, 129151.

  • Hu, M., M. Xue, and K. Brewster, 2006a: 3DVAR and cloud analysis with WSR-88D level-II data for the prediction of Fort Worth tornadic thunderstorms. Part I: Cloud analysis and its impact. Mon. Wea. Rev., 134, 675698, doi:10.1175/MWR3092.1.

    • Search Google Scholar
    • Export Citation
  • Hu, M., M. Xue, J. Gao, and K. Brewster, 2006b: 3DVAR and cloud analysis with WSR-88D level-II data for the prediction of Fort Worth tornadic thunderstorms. Part II: Impact of radial velocity analysis via 3DVAR. Mon. Wea. Rev., 134, 699721, doi:10.1175/MWR3093.1.

    • Search Google Scholar
    • Export Citation
  • Janjić, Z., 2002: Nonsingular implementation of the Mellor-Yamada level 2.5 scheme in the NCEP Meso model. NCEP Office Note 437, 61 pp. [Available online at http://www.emc.ncep.noaa.gov/officenotes/newernotes/on437.pdf.]

  • Johnson, A., X. Wang, F. Kong, and M. Xue, 2013: Object-based evaluation of the impact of horizontal grid spacing on convection-allowing forecasts. Mon. Wea. Rev., 141, 34133425, doi:10.1175/MWR-D-13-00027.1.

    • Search Google Scholar
    • Export Citation
  • Johnson, M., Y. Jung, D. Dawson, and M. Xue, 2016: Comparison of simulated polarimetric signatures in idealized supercell storms using two-moment bulk microphysics schemes in WRF. Mon. Wea. Rev., 144, 971996, doi:10.1175/MWR-D-15-0233.1.

    • Search Google Scholar
    • Export Citation
  • Jung, Y., G. Zhang, and M. Xue, 2008a: Assimilation of simulated polarimetric radar data for a convective storm using the ensemble Kalman filter. Part I: Observation operators for reflectivity and polarimetric variables. Mon. Wea. Rev., 136, 22282245, doi:10.1175/2007MWR2083.1.

    • Search Google Scholar
    • Export Citation
  • Jung, Y., M. Xue, G. Zhang, and J. Straka, 2008b: Assimilation of simulated polarimetric radar data for a convective storm using ensemble Kalman filter. Part II: Impact of polarimetric data on storm analysis. Mon. Wea. Rev., 136, 22462260, doi:10.1175/2007MWR2288.1.

    • Search Google Scholar
    • Export Citation
  • Jung, Y., M. Xue, and G. Zhang, 2010: Simulations of polarimetric radar signatures of a supercell storm using a two-moment bulk microphysics scheme. J. Appl. Meteor. Climatol., 49, 146163, doi:10.1175/2009JAMC2178.1.

    • Search Google Scholar
    • Export Citation
  • Jung, Y., M. Xue, and M. Tong, 2012: Ensemble Kalman filter analyses of the 29–30 May 2004 Oklahoma tornadic thunderstorm using one- and two-moment bulk microphysics schemes, with verification against polarimetric data. Mon. Wea. Rev., 140, 14571475, doi:10.1175/MWR-D-11-00032.1.

    • Search Google Scholar
    • Export Citation
  • Kain, J. S., and Coauthors, 2008: Some practical considerations regarding horizontal resolution in the first generation of operational convection-allowing NWP. Wea. Forecasting, 23, 931952, doi:10.1175/WAF2007106.1.

    • Search Google Scholar
    • Export Citation
  • Kong, F., 2013: 2013 CAPS Spring Forecast Experiment program plan. NOAA/NWS/SPC, 24 pp. [Available online at https://hwt.nssl.noaa.gov/Spring_2013/SpringProgram2013_Plan-v5.pdf.]

  • Kong, F., and Coauthors, 2007: Preliminary analysis on the real-time storm-scale ensemble forecasts produced as a part of the NOAA Hazardous Weather Testbed 2007 Spring Experiment. 22nd Conf. on Weather Analysis and Forecasting/18th Conf. on Numerical Weather Prediction, Salt Lake City, UT, Amer. Meteor. Soc., 3B.2. [Available online at https://ams.confex.com/ams/22WAF18NWP/techprogram/paper_124667.htm.]

  • Kumjian, M. R., and A. V. Ryzhkov, 2008: Polarimetric signatures in supercell thunderstorms. J. Appl. Meteor. Climatol., 47, 19401961, doi:10.1175/2007JAMC1874.1.

    • Search Google Scholar
    • Export Citation
  • Kumjian, M. R., and A. V. Ryzhkov, 2012: The impact of size sorting on the polarimetric radar variables. J. Atmos. Sci., 69, 20422060, doi:10.1175/JAS-D-11-0125.1.

    • Search Google Scholar
    • Export Citation
  • Lakshmanan, V., A. Fritz, T. Smith, K. Hondl, and G. Stumpf, 2007: An automated technique to quality control radar reflectivity data. J. Appl. Meteor. Climatol., 46, 288305, doi:10.1175/JAM2460.1.

    • Search Google Scholar
    • Export Citation
  • Lean, H. W., P. A. Clark, M. Dixon, N. M. Roberts, A. Fitch, R. Forbes, and C. Halliwell, 2008: Characteristics of high-resolution versions of the Met Office Unified Model for forecasting convection over the United Kingdom. Mon. Wea. Rev., 136, 34083424, doi:10.1175/2008MWR2332.1.

    • Search Google Scholar
    • Export Citation
  • Li, X., and J. R. Mecikalski, 2012: Impact of the dual-polarization Doppler radar data on two convective storms with a warm-rain radar forward operator. Mon. Wea. Rev., 140, 21472167, doi:10.1175/MWR-D-11-00090.1.

    • Search Google Scholar
    • Export Citation
  • Lim, K.-S. S., and S.-Y. Hong, 2010: Development of an effective double-moment cloud microphysics scheme with prognostic cloud condensation nuclei (CCN) for weather and climate models. Mon. Wea. Rev., 138, 15871612, doi:10.1175/2009MWR2968.1.

    • Search Google Scholar
    • Export Citation
  • Lin, Y.-L., R. D. Farley, and H. D. Orville, 1983: Bulk parameterization of the snow field in a cloud model. J. Climate Appl. Meteor., 22, 10651092, doi:10.1175/1520-0450(1983)022<1065:BPOTSF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Mahale, V., G. Zhang, and M. Xue, 2014: Fuzzy logic classification of S-band polarimetric radar echoes to identify three-body scattering and improve data quality. J. Appl. Meteor. Climatol., 53, 20172033, doi:10.1175/JAMC-D-13-0358.1.

    • Search Google Scholar
    • Export Citation
  • Mansell, E. R., 2010: On sedimentation and advection in multimoment bulk microphysics. J. Atmos. Sci., 67, 30843094, doi:10.1175/2010JAS3341.1.

    • Search Google Scholar
    • Export Citation
  • Maxwell-Garnett, J. C., 1904: Colors in metal glasses and in metallic films. Philos. Trans. Roy. Soc. London, 203A, 385420, doi:10.1098/rsta.1904.0024.

    • Search Google Scholar
    • Export Citation
  • Mellor, G. L., and T. Yamada, 1982: Development of a turbulence closure model for geophysical fluid problems. Rev. Geophys., 20, 851875, doi:10.1029/RG020i004p00851.

    • Search Google Scholar
    • Export Citation
  • Milbrandt, J. A., and M. K. Yau, 2005a: A multimoment bulk microphysics parameterization. Part I: Analysis of the role of the spectral shape parameter. J. Atmos. Sci., 62, 30513064, doi:10.1175/JAS3534.1.

    • Search Google Scholar
    • Export Citation
  • Milbrandt, J. A., and M. K. Yau, 2005b: A multimoment bulk microphysics parameterization. Part II: A proposed three-moment closure and scheme description. J. Atmos. Sci., 62, 30653081, doi:10.1175/JAS3535.1.

    • Search Google Scholar
    • Export Citation
  • Mittermaier, M., and N. Roberts, 2010: Intercomparison of spatial forecast verification methods: Identifying skillful spatial scales using the fractions skill score. Wea. Forecasting, 25, 343354, doi:10.1175/2009WAF2222260.1.

    • Search Google Scholar
    • Export Citation
  • Morrison, H., J. A. Curry, and V. I. Khvorostyanov, 2005: A new double-moment microphysics parameterization for application in cloud and climate models. Part I: Description. J. Atmos. Sci., 62, 16651677, doi:10.1175/JAS3446.1.

    • Search Google Scholar
    • Export Citation
  • Morrison, H., G. Thompson, and V. Tatarskii, 2009: Impact of cloud microphysics on the development of trailing stratiform precipitation in a simulated squall line: Comparison of one- and two-moment schemes. Mon. Wea. Rev., 137, 9911007, doi:10.1175/2008MWR2556.1.

    • Search Google Scholar
    • Export Citation
  • NWS, 2014: The tornado outbreak of May 20, 2013. NOAA/National Weather Service. [Available online at https://www.weather.gov/oun/events-20130520.]

  • Park, H. S., A. V. Ryzhkov, D. S. Zrnić, and K.-E. Kim, 2009: The hydrometeor classification algorithm for the polarimetric WSR-88D: Description and application to an MCS. Wea. Forecasting, 24, 730748, doi:10.1175/2008WAF2222205.1.

    • Search Google Scholar
    • Export Citation
  • Putnam, B. J., M. Xue, Y. Jung, N. Snook, and G. Zhang, 2014: The analysis and prediction of microphysical states and polarimetric radar variables in a mesoscale convective system using double-moment microphysics, multinetwork radar data, and the ensemble Kalman filter. Mon. Wea. Rev., 142, 141162, doi:10.1175/MWR-D-13-00042.1.

    • Search Google Scholar
    • Export Citation
  • Roberts, N., 2008: Assessing the spatial and temporal variation in the skill of precipitation forecasts from an NWP model. Meteor. Appl., 15, 163169, doi:10.1002/met.57.

    • Search Google Scholar
    • Export Citation
  • Roberts, N., and H. W. Lean, 2008: Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Mon. Wea. Rev., 136, 7897, doi:10.1175/2007MWR2123.1.

    • Search Google Scholar
    • Export Citation
  • ROC, 2013: WSR-88D dual polarization deployment progress. NOAA/Radar Operations Center, 6 pp. [Available online at http://www.roc.noaa.gov/WSR88D/PublicDocs/DualPol/DPstatus.pdf.]

  • Rogers, E., and Coauthors, 2009: The NCEP North American Mesoscale Modeling System: Recent changes and future plans. Preprints, 23rd Conf. on Weather Analysis and Forecasting/19th Conf. on Numerical Weather Prediction, Omaha, NE, Amer. Meteor. Soc., 2A.4. [Available online at https://ams.confex.com/ams/23WAF19NWP/techprogram/paper_154114.htm.]

  • Ryzhkov, A., and D. S. Zrnić, 1996: Assessment of rainfall measurement that uses specific differential phase. J. Appl. Meteor., 35, 20802090, doi:10.1175/1520-0450(1996)035<2080:AORMTU>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Schwartz, C., and Coauthors, 2009: Next-day convection-allowing WRF model guidance: A second look at 2-km versus 4-km grid spacing. Mon. Wea. Rev., 137, 33513372, doi:10.1175/2009MWR2924.1.

    • Search Google Scholar
    • Export Citation
  • Skamarock, W., and Coauthors, 2008: A description of the Advanced Research WRF version 3. NCAR Tech. Note NCAR/TN-475+STR, 113 pp., doi:10.5065/D68S4MVH.

  • Smith, T., and Coauthors, 2016: Multi-radar multi-sensor (MRMS) severe weather and aviation products: Initial operating capabilities. Bull. Amer. Meteor. Soc., 97, 16171630, doi:10.1175/BAMS-D-14-00173.1.

    • Search Google Scholar
    • Export Citation
  • SPC, 2014a: SPC filtered storm reports for 5/19/2013. NOAA/NWS/Storm Prediction Center. [Available online at http://www.spc.noaa.gov/climo/reports/130519_rpts.html.]

  • SPC, 2014b: SPC filtered storm reports for 5/20/2013. NOAA/NWS/Storm Prediction Center. [Available online at http://www.spc.noaa.gov/climo/reports/130520_rpts.html.]

  • Sun, J., and N. A. Crook, 2001: Real-time low-level wind and temperature analysis using single WSR-88D data. Wea. Forecasting, 16, 117132, doi:10.1175/1520-0434(2001)016<0117:RTLLWA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Thompson, G., P. R. Field, R. M. Rasmussen, and W. D. Hall, 2008: Explicit forecasts of winter precipitation using an improved bulk microphysics scheme. Part II: Implementation of a new snow parameterization. Mon. Wea. Rev., 136, 50955115, doi:10.1175/2008MWR2387.1.

    • Search Google Scholar
    • Export Citation
  • Ulbrich, C. W., 1983: Natural variations in the analytical form of the raindrop size distributions. J. Appl. Meteor. Climatol., 22, 17641775, doi:10.1175/1520-0450(1983)022<1764:NVITAF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Van Weverberg, K., A. M. Vogelmann, H. Morrison, and J. A. Milbrandt, 2012: Sensitivity of idealized squall-line simulations to the level of complexity used in two-moment bulk microphysics schemes. Mon. Wea. Rev., 140, 18831907, doi:10.1175/MWR-D-11-00120.1.

    • Search Google Scholar
    • Export Citation
  • Vivekanandan, J., W. M. Adams, and V. N. Bringi, 1991: Rigorous approach to polarimetric radar modeling of hydrometeor orientation distributions. J. Appl. Meteor., 30, 10531063, doi:10.1175/1520-0450(1991)030<1053:RATPRM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Wacker, U., and A. Seifert, 2001: Evolution of rain water profiles resulting from pure sedimentation: Spectral vs. parameterized description. Atmos. Res., 58, 1939, doi:10.1016/S0169-8095(01)00081-3.

    • Search Google Scholar
    • Export Citation
  • Weiss, S. J., and Coauthors, 2007: The NOAA Hazardous Weather Testbed: Collaborative testing of ensemble and convection-allowing WRF models and subsequent transfer to operations at the Storm Prediction Center. 22nd Conf. on Weather Analysis and Forecasting/18th Conf. on Numerical Weather Prediction, Salt Lake City, UT, Amer. Meteor. Soc., 6B.4. [Available online at https://ams.confex.com/ams/22WAF18NWP/techprogram/paper_124772.htm.]

  • Wheatley, D. M., N. Yussouf, and D. J. Stensrud, 2014: Ensemble Kalman filter analyses and forecasts of a severe mesoscale convective system using different choices of microphysics schemes. Mon. Wea. Rev., 142, 32433263, doi:10.1175/MWR-D-13-00260.1.

    • Search Google Scholar
    • Export Citation
  • Xue, M., D.-H. Wang, J.-D. Gao, K. Brewster, and K. K. Droegemeier, 2003: The Advanced Regional Prediction System (ARPS), storm-scale numerical weather prediction and data assimilation. Meteor. Atmos. Phys., 82, 139170, doi:10.1007/s00703-001-0595-6.

    • Search Google Scholar
    • Export Citation
  • Xue, M., M. Tong, and K. K. Droegemeier, 2006: An OSSE framework based on the ensemble square-root Kalman filter for evaluating impact of data from radar networks on thunderstorm analysis and forecast. J. Atmos. Oceanic Technol., 23, 4666, doi:10.1175/JTECH1835.1.

    • Search Google Scholar
    • Export Citation
  • Xue, M., and Coauthors, 2007: CAPS realtime storm-scale ensemble and high-resolution forecasts as part of the NOAA Hazardous Weather Testbed 2007 spring experiment. 22nd Conf. on Weather Analysis and Forecasting/18th Conf. on Numerical Weather Prediction, Salt Lake City, UT, Amer. Meteor. Soc., 3B.1. [Available online at https://ams.confex.com/ams/22WAF18NWP/techprogram/paper_124587.htm.]

  • Xue, M., F. Kong, K. W. Thomas, J. Gao, Y. Wang, K. Brewster, and K. K. Droegemeier, 2013: Prediction of convective storms at convection-resolving 1-km resolution over continental United States with radar data assimilation: An example case of 26 May 2008 and precipitation forecasts from spring 2009. Adv. Meteor., 2013, 259052, doi:10.1155/2013/259052.

    • Search Google Scholar
    • Export Citation
  • Zhang, G., M. Xue, Q. Cao, and D. Dawson, 2008: Diagnosing the intercept parameter for exponential raindrop size distribution based on video disdrometer observations. J. Appl. Meteor. Climatol., 47, 29832992, doi:10.1175/2008JAMC1876.1.

    • Search Google Scholar
    • Export Citation
1

The rimed ice category in the Morrison scheme can be switched to represent either graupel or hail. In this study it was represented as graupel.

Save
  • Brandes, E. A., G. Zhang, and J. Vivekanandan, 2002: Experiments in rainfall estimation with a polarimetric radar in a subtropical environment. J. Appl. Meteor., 41, 674685, doi:10.1175/1520-0450(2002)041<0674:EIREWA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Bringi, V. N., and V. Chandrasekar, 2001: Polarimetric Doppler Weather Radar. Cambridge University Press, 636 pp.

  • Bryan, G. H., and H. Morrison, 2012: Sensitivity of a simulated squall line to horizontal resolution and parameterization of microphysics. Mon. Wea. Rev., 140, 202225, doi:10.1175/MWR-D-11-00046.1.

    • Search Google Scholar
    • Export Citation
  • Bryan, G. H., J. C. Wyngaard, and J. M. Fritsch, 2003: Resolution requirements for the simulation of deep moist convection. Mon. Wea. Rev., 131, 23942416, doi:10.1175/1520-0493(2003)131<2394:RRFTSO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Chen, F., and J. Dudhia, 2001: Coupling an advanced land surface–hydrology model with the Penn State–NCAR MM5 modeling system. Part I: Model implementation and sensitivity. Mon. Wea. Rev., 129, 569585, doi:10.1175/1520-0493(2001)129<0569:CAALSH>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Cintineo, R., J. A. Otkin, M. Xue, and F. Kong, 2014: Evaluating the performance of planetary boundary layer and cloud microphysical parameterization schemes in convection-permitting ensemble forecasts using synthetic GOES-13 satellite observations. Mon. Wea. Rev., 142, 163182, doi:10.1175/MWR-D-13-00143.1.

    • Search Google Scholar
    • Export Citation
  • Clark, A. J., and Coauthors, 2012: An overview of the 2010 Hazardous Weather Testbed Experimental Forecast Program Spring Experiment. Bull. Amer. Meteor. Soc., 93, 5574, doi:10.1175/BAMS-D-11-00040.1.

    • Search Google Scholar
    • Export Citation
  • Dawson, D. T., II, E. R. Mansell, Y. Jung, L. J. Wicker, M. R. Kumjian, and M. Xue, 2014: Low-level ZDR signatures in supercell forward flanks: The role of size sorting and melting of hail. J. Atmos. Sci., 71, 276299, doi:10.1175/JAS-D-13-0118.1.

    • Search Google Scholar
    • Export Citation
  • Du, J., J. McQueen, G. DiMego, Z. Toth, D. Jovic, B. Zhou, and H. Chuang, 2006: New dimension of NCEP Short-Range Ensemble Forecasting (SREF) system: Inclusion of WRF members. Preprints, Expert Team Meeting on Ensemble Prediction System, Exeter, United Kingdom, WMO. [Available online at https://www.wmo.int/pages/prog/www/DPFS/Meetings/ET-EPS_Exeter2006/Doc6(5).pdf.]

  • Ebert, E. E., 2008: Fuzzy verification of high-resolution gridded forecasts: A review and proposed framework. Meteor. Appl., 15, 5164, doi:10.1002/met.25.

    • Search Google Scholar
    • Export Citation
  • Fritsch, J. M., and G. S. Forbes, 2001: Mesoscale convective systems. Severe Convective Storms, Meteor. Monogr., No. 50, Amer. Meteor. Soc., 323–357.

  • Gao, J.-D., M. Xue, K. Brewster, and K. K. Droegemeier, 2004: A three-dimensional variational data analysis method with recursive filter for Doppler radars. J. Atmos. Oceanic Technol., 21, 457469, doi:10.1175/1520-0426(2004)021<0457:ATVDAM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hodur, R. M., 1997: The Naval Research Laboratory’s Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS). Mon. Wea. Rev., 125, 14141430, doi:10.1175/1520-0493(1997)125<1414:TNRLSC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hong, S.-Y., and J.-O. J. Lim, 2006: The WRF single-moment 6-class microphysics scheme (WSM6). J. Korean Meteor. Soc., 42, 129151.

  • Hu, M., M. Xue, and K. Brewster, 2006a: 3DVAR and cloud analysis with WSR-88D level-II data for the prediction of Fort Worth tornadic thunderstorms. Part I: Cloud analysis and its impact. Mon. Wea. Rev., 134, 675698, doi:10.1175/MWR3092.1.

    • Search Google Scholar
    • Export Citation
  • Hu, M., M. Xue, J. Gao, and K. Brewster, 2006b: 3DVAR and cloud analysis with WSR-88D level-II data for the prediction of Fort Worth tornadic thunderstorms. Part II: Impact of radial velocity analysis via 3DVAR. Mon. Wea. Rev., 134, 699721, doi:10.1175/MWR3093.1.

    • Search Google Scholar
    • Export Citation
  • Janjić, Z., 2002: Nonsingular implementation of the Mellor-Yamada level 2.5 scheme in the NCEP Meso model. NCEP Office Note 437, 61 pp. [Available online at http://www.emc.ncep.noaa.gov/officenotes/newernotes/on437.pdf.]

  • Johnson, A., X. Wang, F. Kong, and M. Xue, 2013: Object-based evaluation of the impact of horizontal grid spacing on convection-allowing forecasts. Mon. Wea. Rev., 141, 34133425, doi:10.1175/MWR-D-13-00027.1.

    • Search Google Scholar
    • Export Citation
  • Johnson, M., Y. Jung, D. Dawson, and M. Xue, 2016: Comparison of simulated polarimetric signatures in idealized supercell storms using two-moment bulk microphysics schemes in WRF. Mon. Wea. Rev., 144, 971996, doi:10.1175/MWR-D-15-0233.1.

    • Search Google Scholar
    • Export Citation
  • Jung, Y., G. Zhang, and M. Xue, 2008a: Assimilation of simulated polarimetric radar data for a convective storm using the ensemble Kalman filter. Part I: Observation operators for reflectivity and polarimetric variables. Mon. Wea. Rev., 136, 22282245, doi:10.1175/2007MWR2083.1.

    • Search Google Scholar
    • Export Citation
  • Jung, Y., M. Xue, G. Zhang, and J. Straka, 2008b: Assimilation of simulated polarimetric radar data for a convective storm using ensemble Kalman filter. Part II: Impact of polarimetric data on storm analysis. Mon. Wea. Rev., 136, 22462260, doi:10.1175/2007MWR2288.1.

    • Search Google Scholar
    • Export Citation
  • Jung, Y., M. Xue, and G. Zhang, 2010: Simulations of polarimetric radar signatures of a supercell storm using a two-moment bulk microphysics scheme. J. Appl. Meteor. Climatol., 49, 146163, doi:10.1175/2009JAMC2178.1.

    • Search Google Scholar
    • Export Citation
  • Jung, Y., M. Xue, and M. Tong, 2012: Ensemble Kalman filter analyses of the 29–30 May 2004 Oklahoma tornadic thunderstorm using one- and two-moment bulk microphysics schemes, with verification against polarimetric data. Mon. Wea. Rev., 140, 14571475, doi:10.1175/MWR-D-11-00032.1.

    • Search Google Scholar
    • Export Citation
  • Kain, J. S., and Coauthors, 2008: Some practical considerations regarding horizontal resolution in the first generation of operational convection-allowing NWP. Wea. Forecasting, 23, 931952, doi:10.1175/WAF2007106.1.

    • Search Google Scholar
    • Export Citation
  • Kong, F., 2013: 2013 CAPS Spring Forecast Experiment program plan. NOAA/NWS/SPC, 24 pp. [Available online at https://hwt.nssl.noaa.gov/Spring_2013/SpringProgram2013_Plan-v5.pdf.]

  • Kong, F., and Coauthors, 2007: Preliminary analysis on the real-time storm-scale ensemble forecasts produced as a part of the NOAA Hazardous Weather Testbed 2007 Spring Experiment. 22nd Conf. on Weather Analysis and Forecasting/18th Conf. on Numerical Weather Prediction, Salt Lake City, UT, Amer. Meteor. Soc., 3B.2. [Available online at https://ams.confex.com/ams/22WAF18NWP/techprogram/paper_124667.htm.]

  • Kumjian, M. R., and A. V. Ryzhkov, 2008: Polarimetric signatures in supercell thunderstorms. J. Appl. Meteor. Climatol., 47, 19401961, doi:10.1175/2007JAMC1874.1.

    • Search Google Scholar
    • Export Citation
  • Kumjian, M. R., and A. V. Ryzhkov, 2012: The impact of size sorting on the polarimetric radar variables. J. Atmos. Sci.,