1. Introduction
Echo-top height is defined as the maximum height of a standard value of radar reflectivity in a vertical column at a given location. It has been estimated from weather radars for a wide range of purposes, especially to determine thunderstorm structures (Smalley et al. 2003) and areas of potential turbulence in anvils (Smith et al. 2016), to study electrical discharges (MacGorman et al. 2017) or diagnose hail (Held 1978; Amburn and Wolf 1997; Delobbe and Holleman 2006). It is a critical input for aviation weather forecasting (Evans et al. 2004). In 2015, 31% of air traffic delays in the United States were attributed to weather, with weather’s share of National Airspace System delays adding up to 53%, costing air carriers $1,400–$4,500 per hour (Bureau of Transportation Statistics 2015; Schoor 2016). Accurate measurements of echo-top heights allow air traffic controllers to route aircraft away from aviation weather hazards more effectively while limiting their consumption of fuel and delays for passengers.
Uncertainties associated with echo-top height estimates originate to a large extent from weather radar sampling. In the United States, the National Weather Service’s Next Generation Weather Radar (NEXRAD) network samples the atmosphere with S-band Weather Surveillance Radar-1988 Doppler (WSR-88D). When using a single radar, the limited number of radar elevations angles induces misestimations of echo-top heights. The estimation of echo-top heights close to the radar suffers from data voids above the highest elevation angle, referred to as the cone of silence. At long distances from the radar (typically greater than 70 km), the beam broadening decreases the accuracy of echo-top heights estimation. Maddox et al. (1999) showed the benefit of using simultaneous observations from at least two adjacent radars to fill sampling related gaps and reduce uncertainties in estimated heights. This finding led to the design of an echo-top height algorithm in the National Mosaic and Multi-Sensor Quantitative Precipitation Estimation (NMQ; Zhang et al. 2011) covering the conterminous United States (CONUS).
Single WSR-88D echo-top height algorithms use 18 dBZ as the default standard value of reflectivity (Lakshmanan et al. 2013; Office of the Federal Coordinator for Meteorological Services and Supporting Research 2017), which possibly explains why the 18-dBZ threshold was chosen in the first place in the NMQ system (Zhang et al. 2011).
The NMQ system was updated to the Multi-Radar Multi-Sensor (MRMS) system in September 2014 at NOAA’s National Centers for Environmental Prediction (NCEP) following the upgrade of the NEXRAD network to the dual-polarimetric technology (Zhang et al. 2016). The MRMS system combines information from all ground-based radars composing the NEXRAD network, mosaics reflectivity data onto a common 3D grid to derive several meteorological products including the 18-dBZ echo-top heights (Smith et al. 2016; NOAA Virtual Lab 2023).
The advent of space-based radar with the Tropical Rainfall Measurement Mission (TRMM) in 1997, followed in 2014 by the Dual-Frequency Phased Array Precipitation Radar (DPR) on board NASA’s Global Precipitation Measurement (GPM) Core Observatory satellite opened new perspectives for improving the monitoring and quantification of echo-top height (Hou et al. 2014). Because the vantage point of space is more suited for the estimation of echo-top heights than ground-based radars, spaceborne radar data provide a unique opportunity for systematic evaluation of ground-based echo-top heights.
The goal of this study is to compare over the CONUS the 18-dBZ echo-top heights from MRMS with those from the Ku-band Precipitation Radar (KuPR) of the DPR, considered as the reference. Here in section 1, MRMS and KuPR overviews are presented. The datasets and the method are detailed in section 2. In section 3, the results of a 19-month comparison are presented. Specifically, the effects of the closest radar volume coverage pattern and the use of the automated volume scan evaluation and termination (AVSET) mode on differences between ground-based and spaceborne radars are discussed.
2. Dataset and methods
a. KuPR on the GPM Core Observatory
KuPR is a radar on board the GPM Core Observatory. The GPM satellite has a circular, non-sun-synchronous orbit with an inclination of 65° to the equator at a mean altitude of 407 km (Hou et al. 2014). KuPR is one of the instruments of the DPR developed by the Japan Aerospace Exploration Agency (JAXA) and the Japan National Institute of Information and Communications Technology. It operates at a frequency of 13.6 GHz and acquires data in a swath of 245 km with a vertical resolution of 250 m and a horizontal resolution at nadir of 5 km. With a minimum detectable reflectivity of 18 dBZ (Hou et al. 2014), the radar provides a spaceborne estimate of the 18-dBZ echo-top height.
In this study the KuPR is considered as the reference because of its vertical sampling. The uncertainty associated with echo-top estimates depends on the vertical resolution of the instrument (250 m at nadir), whereas ground-based radars are subject to beam broadening (the radar beamwidth is about 1 km at 60 km from the radar) because of their horizontal sampling as well as coverage issues. The KuPR provides a higher vertical resolution than the MRMS 3D grid, especially in the upper parts of the atmosphere (section 2b). Another advantage of the KuPR is that the same radar consistently observes echo tops over the CONUS without any calibration difference or failure that can affect ground-based radars in such a network.
To distinguish events in terms of typology, we use the DPR precipitation type classification (Awaka et al. 2016). This typology conditions the drop size distribution model used to estimate surface precipitation rates. Two distinctive precipitation types exist: stratiform and convective. Stratiform precipitation features weak to moderate rain rates, a generally wide spatial extension and, depending on atmospheric conditions, potentially a bright band in the radar echoes (Kirstetter et al. 2013). The detection of this bright band in the radar profile can be used to determine the stratiform precipitation type. Convective precipitation features moderate to large rain rates, a generally significant vertical extension, and a variable horizontal extension.
b. MRMS
The MRMS is an operational system dedicated to quantitative precipitation estimation, severe weather monitoring and aviation hazards guidance over the CONUS (Zhang et al. 2016). Developed at the NOAA/National Severe Storms Laboratory and the University of Oklahoma, it is operational at the NCEP of the National Weather Service. This system integrates data from about 180 operational radars from the NEXRAD and the Environment and Climate Change Canada networks, the National Lightning Detection Network, surface terrain elevation information and hourly analyses from the Rapid Refresh model. Radar data are blended to create a 3D volume of quality-controlled reflectivity data. Radars and lightning detectors provide continuous input data streams, allowing a minimal temporal resolution of 2 min and a horizontal resolution of 0.01° in both latitude and longitude. The vertical grid spacing of the radar reflectivity field is 0.25 km from 0.5 to 3 km above the mean sea level (MSL), 0.5 km from 3 to 9 km MSL, and 1 km from 9 to 19 km MSL, for a total of 33 vertical levels. MRMS provides echo-top heights over a domain extending between 20° and 55°N in latitude and between 130° and 60°W in longitude.
Some differences between the operational method implemented in MRMS and the one described by Lakshmanan et al. (2013) are evident. First is the use of multiple radars in MRMS, whereas the Lakshmanan’s scheme has been tested on single radars. Another difference is that in MRMS interpolated values of reflectivity are processed, while in Lakshmanan’s scheme the echo-top heights are computed based on observed (i.e., noninterpolated) radar measurements. Note that in MRMS the interpolation scheme is applied perpendicular to the beam (the maximum elevation angle is 19.5°), so some 3D reflectivities are interpolated onto neighboring vertical columns. For each kilometer separating the voxel from the center of the beam in the vertical direction, the maximum shift in the horizontal direction is approximately 0.35 km, which can lead to errors in echo-top height estimates.
c. Matching between sensors
Because the horizontal resolution of MRMS and KuPR differ, a horizontal matching between the two radar products is required to compare them. To make calculations simpler, the KuPR tile is assumed to be a constant 5-km-diameter circle, regardless of the radar beam off-nadir inclination angle [Fig. 2.2.2 in Iguchi et al. (2021)], whereas the MRMS tile has a 0.01° horizontal resolution. A MRMS echo-top height is built at the resolution of KuPR by weighting each MRMS value, according to the KuPR antenna energy pattern following the procedure described in Kirstetter et al. (2012). It gives more weight to the pixels close to the center of the beam and less weight to the pixels at the edges.
The weights ωi are derived from the two-way normalized power-gain function of the KuPR antenna f (assumed to be Gaussian) and the −3-dB beamwidth θ0. Thus, each ωi is computed over the domain θmesh corresponding to the MRMS mesh ai. If more than 10% of the MRMS tiles included in the KuPR footprint indicate missing values, none of the MRMS and KuPR observations are further considered in the analysis.
As for the DPR, stratiform and convective precipitation types from the MRMS precipitation type classification are computed for each echo top. Hail, convective rain, and tropical–convective rain mix types are combined into the convective category. Warm stratiform rain, cool stratiform rain and tropical–stratiform rain mix types are combined into the stratiform type.
3. Results
a. Preliminary study
A preliminary study of the dataset (Fig. 1) revealed that there is a strong relationship between the KuPR and MRMS echo-top heights, most of the points being gathered around the identity line. A linear regression gives a slope of 1.067 and a y intercept of −0.031 km with a coefficient of determination of 0.712 indicating a positive linear relationship at a high confidence level. For high echo-top heights (>10 km) a majority of points lay above the identity line, which indicates a systematic overestimation of the echo-top height given by MRMS relative to KuPR. One can notice a pattern of points gathered close to MRMS 3D levels, specifically every kilometer above 9 km. This pattern is due to the MRMS echo-top height algorithm: in Eq. (2), the assignment of −25 dBZ for R1 in case of a missing value or a value under the detection threshold results in an echo-top height close to z2, the 3D level of MRMS.
MRMS 18-dBZ echo-top height as a function of KuPR 18-dBZ echo-top height for the 19 months (a) before and (b) after filtering. Normalized two-dimensional kernel density estimation filled contours are superimposed when the density of observations is high.
Citation: Journal of Applied Meteorology and Climatology 62, 8; 10.1175/JAMC-D-22-0146.1
The ability to identify MRMS artifacts with statistical parameters has been investigated. The local zenith angle (the angle between the zenith and the KuPR radar beam) seems to play no role in the differences in echo-top heights between MRMS and KuPR, even when the angle is close to 19° (not shown).
The MRMS domain includes territories such as Mexico, Canada, and most of the Caribbean islands with degraded WSR-88D coverage or covered by other radar models. To ensure the results are not affected by this effect, only the overland CONUS observations are kept. Figure 2 displays echo-top height differences between MRMS and KuPR over the 19-month period over CONUS. One can see in Fig. 2a the significant differences associated with points far away from the radars, especially over the ocean. A potential factor impacting the accuracy of echo-top height estimation is the distance to the radar, as pointed out by Lakshmanan et al. (2013). Since MRMS combine radar observations, predictors involving the distance to nearby radars and the number of radars covering each observation were investigated. However, these parameters do not appear to be good predictors of echo-top height differences (not shown). There are several explanations for this finding. First, Howard et al. (1997) stated that even with one single radar, the echo-top height error does not grow linearly with distance. Second, each MRMS voxel value is computed by weighting the three closest radars, always the same, even if one of them is down. Without knowing the state of each radar (operational or in maintenance), we cannot truly know which radars are used for each voxel. Third, the MRMS reflectivity merger blends radars operating with different calibrations and different scanning schedules for the volume coverage pattern (VCP). Calibration and temporal mismatches add uncertainties in the echo-top height estimation. The reflectivity quality control (QC) is not perfect; that is, not all nonprecipitating echoes near 18 dBZ are always eliminated, and masks may also degrade the quality of the echo-top height.
Difference in 18-dBZ echo-top height between MRMS and KuPR for the 19 months (a) before and (b) after filtering.
Citation: Journal of Applied Meteorology and Climatology 62, 8; 10.1175/JAMC-D-22-0146.1
An interesting finding is that a substantial proportion of MRMS artifacts are associated with low standard deviation of the MRMS echo-top heights within the KuPR footprint, both in convective and stratiform situations. Low standard deviation indicates that the 0.01° resolution MRMS echo-top heights that compose the KuPR footprint are nearly identical, which is very unlikely over an ∼20-km2 area. Most of these low standard deviations are the result of values missing or below the reflectivity threshold, associated with measurements taken at long distance from the radar. It is confirmed with the KuPR land type classification that most of these cases occur over water. By comparing the distribution of differences in echo-top heights between MRMS and KuPR with the standard deviation values (not shown), a threshold of 0.2 km is identified to filter out unrepresentative MRMS echo-top heights associated with these low standard deviations. The large sample size of MRMS–KuPR data pairs allows us to apply this conservative threshold. The filter applied on the 1 496 928 points consists in keeping data pairs
-
over land of the CONUS (995 696 remaining pairs);
-
both below 19 km, because the last level of MRMS is 19 km (994 941 remaining pairs);
-
whose standard deviation of the MRMS echo-top heights within the KuPR footprint is above 0.2 km (951 046 remaining pairs);
-
outside radar cones of silence, to avoid this effect—even if cones of silence are filled with other radars in MRMS. The filtering is performed such that all of the points above the radar elevation angle of 19.5° are eliminated.
Over the 19-month period, 919 966 observations remain (Figs. 1b and 2b). Note that the overestimation of echo-top heights above 9 km by MRMS (Figs. 1a and 2a) is significantly mitigated. A linear regression gives a slope of 1.014 and a y intercept of 0.138 km. The coefficient of determination increases from 0.712 before filtering to 0.728 after filtering, showing its benefit.
b. Climatology of echo-top heights between June 2014 and December 2015
KuPR and MRMS distributions of echo-top heights (Fig. 3) show an overall agreement with similar modes around 5.5 km, indicating a predominance of precipitation systems with moderate vertical extent, and tails highlighting the occurrences of deeper events. Yet the ground-based and spaceborne radars have few differences in their echo-top height distributions. The MRMS distribution is wider and has a longer tail than KuPR. The MRMS distribution is flatter for medium heights (4–7 km), whereas the KuPR distribution is more peaked around the mode, suggesting a better separation of shallow versus deep precipitation from space. MRMS records more frequent echo tops at low heights (0–3.5 km), which may reflect the ability of ground-based radars to detect shallow events, while spaceborne radars are affected by the blind range (Smalley et al. 2017). The MRMS distribution also displays more frequent echo-top heights between 7.5 and 10 km, and above 11 km, which may be caused by uncertainties in ground radar reflectivity measurements, gaps in radar coverage, beam broadening, Cartesian regridding, and the interpolation procedure to compute echo-top heights (section 2b). Note that filtering MRMS values resulted in the diminution of frequencies in the tail of the MRMS distribution, better agreeing with the KuPR distribution.
Distribution of 18-dBZ echo-top height, for the 19 months after filtering, for MRMS (red) and KuPR (blue); the area where the two histograms are superimposed is in violet.
Citation: Journal of Applied Meteorology and Climatology 62, 8; 10.1175/JAMC-D-22-0146.1
Figure 4 shows with monthly boxplots that both MRMS and KuPR display a seasonal trend of echo-top heights over the United States. Echo-top heights reach a maximum (median between 7 and 9 km) during the meteorological summer (June, July, August) with a peak in July. The lowest echo tops are observed between November and March. During these months, the median echo top reaches heights between 4 and 5.5 km. Generally higher instability allows for the formation of clouds with large vertical extension during the summer, while the more stable atmosphere in winter results in shallower clouds. This explains why the median echo-top height is higher in the summer than during the winter. The amplitude of the MRMS annual cycle is higher than that of KuPR by 500–750 m in median. Echo tops are overestimated by MRMS between June and September and underestimated between November and March, as can be seen by the shift of the MRMS distribution toward higher values with respect to KuPR in summer (e.g., higher MRMS median and third quartile) and toward lower values in winter (e.g., lower MRMS median and first quartile). Overestimations in summer reach a maximum of 300 m in median in 2014 but exceed 500 m during these four months in 2015. During winter, the underestimations in median remain within −450 m. The seasonal dependency of both the systematic differences and the spread of the distribution of differences (not shown), which is higher during summer than during winter, can be noted.
Distributions of 18-dBZ echo-top heights measured by MRMS and KuPR for the 19 months. Each box shows the median and the first and third quartiles of each bin, and its width is proportional to the sample size in the bin. Dotted lines show the first and ninth deciles.
Citation: Journal of Applied Meteorology and Climatology 62, 8; 10.1175/JAMC-D-22-0146.1
To investigate the links between differences between MRMS and KuPR and clouds involved, convective and stratiform cases are separated. A convective case is identified both by KuPR as convective and by MRMS as convective or “convective and stratiform.” This last category means that at least one MRMS tile inside the KuPR footprint is detected as convective. Stratiform cases are indicated by both MRMS and KuPR as stratiform. Table 1 indicates the corresponding samples in bold.
Contingency table of the rain-type classification given by MRMS (rows) and KuPR (columns) for the filtered dataset. The samples corresponding to the cases used to separate “convective” and “stratiform” cases, as defined in the main text, are in boldface type.
For convective cases, positive median differences are observed for all months (Fig. 5). Over the 19 months, the mean difference is 1.06 km and the median difference is 0.90 km, indicating a systematic overestimation of MRMS echo-top heights. During 7 of the 19 months, more than 75% of the convective echo tops are overestimated by MRMS. For stratiform cases, the median is above zero mostly during summer months (7 months over 19) and below zero the rest of the time. Over the 19 months, the mean difference is 0.13 km while the median difference is −0.05 km.
Distributions of differences in 18-dBZ echo-top height between MRMS and KuPR for convective and stratiform cases as a function of the month for the 19 months. Boxes and lines are as in Fig. 4.
Citation: Journal of Applied Meteorology and Climatology 62, 8; 10.1175/JAMC-D-22-0146.1
One can notice that the spread of the differences, indicated by the interquartile range, is higher for convective cases (2.35 km) than for stratiform ones (1.47 km). For stratiform cases, in summer, the median–ninth decile interval is around 1.5–2 times the median–first decile interval. This skewness can be explained by misidentifications of both algorithms of stratiform clouds that are in fact stratiform parts of convective clouds (anvils for example).
Figure 6 shows the differences as a function of MRMS echo-top height classes. Overall low echo tops below 5 km (mainly stratiform) are slightly underestimated by MRMS with median differences around −0.5 km and echo tops above 7 km (mainly convective) are overestimated with median differences reaching up to 6.8 km for the highest tops. The conditional biases are similar in the median sense between convective and stratiform cases until 6 km. Between 7 and 10 km, the stratiform median differences increase at a slower pace than the convective median differences. Above 10 km, the difference increases in both cases and at a higher pace for stratiform than convective cases. As mentioned before, higher tops for these stratiform cases may be linked to stratiform parts of convective clouds as cumulonimbus, because only parts of cumulonimbus can produce echo-top heights reaching more than 10 km. The only stratiform clouds able to produce high echo tops are nimbostratus and altostratus, but these clouds rarely exceed 6 to 8 km in height.
Distributions of differences in 18-dBZ echo-top height between MRMS and KuPR as a function of MRMS echo-top height classes for convective and stratiform cases. Boxes and lines are as in Fig. 4, with the sample size in the bin indicated.
Citation: Journal of Applied Meteorology and Climatology 62, 8; 10.1175/JAMC-D-22-0146.1
In looking at the MRMS–KuPR differences as a function of the KuPR echo-top height conditioned on the rain-type classification (Fig. 7), two different behaviors can be noticed. Median differences are near zero below 9 km and point toward underestimation (−0.9–0 km) above 9 km for stratiform cases. On the contrary, for convective cases, MRMS systematically overestimates echo-top heights in median (0.4–2.4 km) with a minimum difference for echo tops heights close to 10 km.
As in Fig. 6, but as a function of KuPR echo-top height classes.
Citation: Journal of Applied Meteorology and Climatology 62, 8; 10.1175/JAMC-D-22-0146.1
c. Influence of the closest radar volume coverage pattern
The impact of the closest radar VCP was investigated on echo-top height differences between MRMS and KuPR (Table 2). In 2014 and 2015, over the CONUS, when precipitation occur, the most used VCP is 12/212 (not shown). The second most used VCP is 21/121/221 particularly used in the eastern and northwestern United States. The least used VCP is 11/211, only used in Texas and by some forecast offices in southeastern or northeastern United States. Forecaster’s choices of VCPs have been very stable between 2014 and 2015 and have depended on weather but also on the forecasters’ habits [Fig. 2 in Kingfield and French (2022)].
Major features of the existing VCPs in 2014/15 for the WSR-88D.
Knowing the time and position of each echo top measurement by both MRMS and KuPR, the closest radar was identified along with its operating VCP and the corresponding last elevation angle at the time of the measurement. A VCP matched dataset, accounting for 99.3% of the filtered dataset was built. Note that the choice of the VCP depends itself on the echo top: Office of the Federal Coordinator for Meteorological Services and Supporting Research (2017) recommendations are to shift the scanning mode from VCP 21/121/221 to VCPs 11/211 or 12/212 in case of deep convection and high echo tops. To study the influence of the VCP on the distributions of MRMS–KuPR echo-top height differences, we separate echo tops by bins of MRMS echo-top height and compare only VCPs 12/212 and 21/121/221, which are used all over the United States more often than the others. As expected, VCP 21/121/221 is more used in stratiform cases (Fig. 8b) in comparison to convective cases (Fig. 8a), and its use decreases with increasing MRMS echo-top height.
Distributions of differences in 18-dBZ echo-top height between MRMS and KuPR as a function of MRMS echo-top height classes when the closest radar VCP is 12/212 (green) and 21/121/221 (yellow) in (a) convective and (b) stratiform cases. Boxes and lines are as in Fig. 6.
Citation: Journal of Applied Meteorology and Climatology 62, 8; 10.1175/JAMC-D-22-0146.1
For convective cases, both VCPs show underestimation of the lowest echo tops (most of the distribution of differences is negative). Both show an upward trend leading to MRMS overestimations by several kilometers for the higher tops. For MRMS echo-top height bins between 7 and 14 km, VCP 21/121/221 displays a systematic overestimation in median (0.4–0.8 km) relative to VCP 12/212. For VCP 12/212, the interdecile stays between 3.3 and 4.3 km for all echo-top classes. However, for VCP 21/121/221, the interdecile range increases with echo-top height, exceeding 4.3 km for tops higher than 12 km. For low echo tops, the spread of the VCP 21/121/221 subset seems to be lower and the underestimations slighter in median than with VCP 12/212. However, one needs to be careful because the VCP 12/212 sample size is 10 times larger than VCP 21/121/221 sample size. A Welsh’s t test is realized to test the equality of the distribution means between both VCPs for each MRMS echo-top height class (Table 3). The assumptions of the test are a normal distribution in each subset. The hypothesis H0 is the equality of the means, and the alternative hypothesis H1 is that VCP 21/121/221 mean is higher than VCP 12/212 mean (one-tailed test). P values lower than 0.001 for the MRMS echo-top heights below 14 km indicates that if H0 was verified, such distributions would be unlikely (less than 0.1% chance). Therefore, except between 14 and 19 km, the test supports H1, the hypothesis that the mean difference for the VCP 21/121/221 is higher than VCP 12/212, with a higher confidence as the p value is lower.
Welsh’s t test of the equality of the MRMS–KuPR difference means between VCPs 12/212 and 21/121/221 for convective and stratiform cases. Sample size is indicated in parentheses.
For stratiform cases, the median MRMS–KuPR difference increases for both VCPs from around −0.8 km for low values of MRMS echo tops to around 3.6 km for the highest MRMS tops (Fig. 8b). For echo tops lower than 5 km, the difference between both VCP in median, interquartile or interdecile ranges is close to zero. When stratiform tops are higher than 6 km, VCP 21/121/221 consistently shows large overestimations compared to VCP 12/212, with a gap of 1.4 km between the two medians for MRMS heights between 9 and 10 km. The difference spread increases with MRMS echo-top height for both subsets. However, there is no evidence of a significant difference in spread, at least in interdecile or interquartile range between the two VCPs. A Welsh’s t test is realized with the same hypotheses as for the convective case (Table 3), except for the bins [0, 3[ (lower mean, as indicated by footnote a) and [3, 4[ (two-tailed test, as indicated by footnote c). Given the huge sample sizes in each bin, we expect lower p values. The p values less than 10−15 support that the VCP 21/121/221 mean is higher than VCP 12/212 in the range 4–19 km; in the range 0–4 km, the departure of the means is not significant.
d. Effect of the AVSET function in VCP 12/212
The AVSET function aims at terminating the current VCP after the radar has scanned all the elevations where significant precipitation signal is available (Chrisman 2009). The drawback is the absence of higher elevation tilts in a volume scan. The benefit of AVSET is to shorten the elapsed time between data collection at low elevation angles. For example, the mean completion time of a VCP 12/212 is 256 s. The AVSET-controlled shortest VCP 12/212 completes a scan in only 190 s with a last elevation angle at 6.4°. Two conditions must be met before the AVSET process terminates a volume scan:
-
the areal coverage of reflectivity above 30 dBZ is less than 30 km2;
-
the areal coverage of reflectivity above 18 dBZ is less than 80 km2 and has not increased by 12 km2 or more since the last volume scan.
When these conditions are met, the AVSET process allows a last tilt above the level where these conditions are met and terminates the volume scan. To study the influence of the AVSET mode on the distributions of differences in echo-top height between MRMS and KuPR, only VCP 12/212 is used to avoid the influence of the VCP shown in section 3c and because it is the most used VCP over the CONUS. In VCP 12/212, the AVSET-controlled shortest VCP with a last tilt of 6.4° is compared with a normal scan.
In convective cases and in VCP 12/212, 70% of the observations are normal scans (Table 4). In 30% of the cases, the last tilt is below 19.5° (AVSET mode) and is 6.4° (AVSET-controlled shortest VCP) in 7% of the cases. For both AVSET and normal modes, the difference in echo-top height between MRMS and KuPR increases with MRMS echo-top height (Fig. 9a). AVSET distribution of differences are shifted toward higher values compared to normal VCP distribution of differences. For echo tops lower than 8 km, the normal VCP shows underestimations of MRMS (the distribution of differences is mostly below zero) whereas in AVSET mode, overestimations are systematic for bins above 5 km. The significance of the differences in mean between both subsets is evaluated by a Welsh’s t test (Table 5). The null hypothesis is the equality of the means, and the alternative hypothesis is that the AVSET-controlled shortest VCP mean is higher than the normal scan mean. Below 14 km, since the p value is lower than 0.001, the test supports a higher mean of differences for AVSET-controlled shortest VCP relative to normal scans.
Number of observations as a function of the last tilt (°; columns) in VCP 12/212.
Distributions of differences in 18-dBZ echo-top height between MRMS and KuPR as a function of MRMS echo-top height classes when the last elevation angle of the closest radar is 6.4° (AVSET-controlled shortest VCP 12/212) and 19.5° (normal VCP 12/212) in (a) convective and (b) stratiform cases. Boxes and lines are as in Fig. 6.
Citation: Journal of Applied Meteorology and Climatology 62, 8; 10.1175/JAMC-D-22-0146.1
As in Table 3, but between cases when the last elevation angle of the closest radar is 6.4° (AVSET-controlled shortest VCP 12/212) and 19.5° (normal VCP 12/212).
For stratiform cases, the last tilt is 19.5° for 71% of the observations in VCP 12/212 and 6.4° in 8% of the cases, a proportion slightly higher than in convective cases (Table 4). As in convective cases, Fig. 9b shows an increasing difference trend for both subsets, and differences are shifted toward higher values in the AVSET mode. In the AVSET mode, differences are closer to zero for MRMS echo-top heights below 6 km but they increase at a faster pace for echo tops higher than 7 km. However, for these tops, the spread in interquartile range or interdecile range is lower in the AVSET mode than in normal scan. With the same hypotheses as in convective cases, the Welsh’s t test gives p values less than 0.001 except for bins in the range 11–19 km (Table 5). The test supports a higher mean of difference for AVSET-controlled shortest VCP compared to normal scans below 11 km, especially for bins between 4 and 11 km whose p values are the lowest.
e. Interpretation of MRMS echo-top biases
Before the advent of MRMS, some single-radar-based products used the center of the radar beam to estimate the echo top. This led to systematic underestimations, especially for high echo tops, because of the elevation-angle sampling voids.
The differences in echo-top heights observed between two VCP strategies show that the echo-top linear interpolation algorithm is not the only cause of overestimations. The algorithm performs interpolations between two vertical levels, separated by 1 km at most, while overestimations reach a few kilometers for the higher tops. This suggests that before computing the echo-top height, the interpolation of single-radar reflectivities into a 3D grid causes overestimations. First, the interpolation extends reflectivity values on the vertical where data voids are important because the weights decrease exponentially at the cube of the angular distance. Then, when the MRMS levels are filled, the echo-top linear interpolation algorithm may increase its height a little more.
Also, for different radars sampling at the same point, the closest radar has more influence. If an echo top is sampled by a radar very close to it, no upward extrapolation is performed from the last elevation where an echo is detected: in that case, the echo-top height given is the top of the beam. Another point is the decreasing number of MRMS 3D levels with height: fewer 3D levels intersecting radar beam centers increases the side effects of interpolation and extrapolation. The wider the voids between tilts are, the more overestimations these processes cause in VCP 21/121/221 relative to VCP 12/212.
However, for stratiform cases, underestimations are observed. They are mostly sampled at radar’s lowest elevation angles, so the phenomenon of vertical extension of the 18-dBZ echo is smaller because the angular separation of elevation angles is lower. Then, at heights below 9 km, MRMS 3D levels are spaced every 500 m and every 250 m below 3 km. Finally, the gradients of reflectivity are smoother in stratiform cases, which limits the side effects of interpolations. A hypothesis to explain underestimations could be a correlation between the occurrence of stratiform clouds of low vertical extension and atmospheric conditions supporting subrefraction of the radar beam (e.g., both strong temperature and dewpoint inversions; Babin 1995).
For AVSET-controlled shortest VCP, overestimations are especially found for high echo tops and are higher than for normal scans. AVSET conditions are tested for all angles above 5°. In VCP 12/212, when the last angle is 6.4°, it indicates that an area of less than 80 km2 of reflectivity above 18 dBZ has been detected at the elevation angle of 5.1°. For several valid precipitation cases this threshold of 80 km2 is not exceeded with a nonzero area of 18-dBZ reflectivity. The gap in height between the center of a 5.1° beam and 6.4° beam is 1 km at 45-km range and 2 km at 90-km range, but the gap between both beams (lower limit of the beam of the 6.4° tilt minus higher limit of the beam of the 5.1° tilt) is only 250 m at 45-km range and 500 m at 90-km range. If the next tilt (6.4° tilt) does not sample a reflectivity above 18 dBZ, the result is unchanged in AVSET or normal mode. If the radar still detects some 18-dBZ reflectivity, however, then, because it is the last elevation angle of the scan, MRMS uses the reflectivity value in a closer neighbor-approach for the higher part of the beam. All MRMS levels included in the higher part of the beam receive the same value of reflectivity. Above 6.4°, the closest radar indicates a missing value, so it weights as 0. If any reflectivity above 18 dBZ remains above, this measure is subject to beam broadening. The AVSET mode is producing an artificial cone of silence when reflectivities are assumed to be insignificant.
For low echo-top heights with AVSET, a reduction of the underestimations is noticed. In this case, AVSET may be beneficial because for these heights, the coverage is the same as a normal VCP, with shorter revisit time periods. Indeed, in VCP 12/212, the AVSET-controlled shortest VCP produces one scan every 190–210 s as compared with one scan every 257–290 s in a normal mode.
4. Conclusions
Uncertainties affect the echo-top height estimates from ground radars. These are mainly due to the radar measurement of reflectivity, beam broadening, Cartesian regridding, and the final interpolation to compute the echo-top height. The MRMS 18-dBZ echo-top heights have been evaluated with respect to the GPM KuPR over the United States. No major bias was assessed between the two products after data matching and filtering. The MRMS distribution of echo tops heights is wider compared to KuPR. An annual cycle of the differences is observed, driven by stratiform events for which echo tops are slightly underestimated. Conversely, convective echo tops are mainly overestimated. Underestimations affect preferentially low MRMS echo tops whereas overestimations affect higher MRMS tops. The investigation of these systematic biases for different radar scanning strategies shows that VCPs with fewer tilts and greater voids between the highest tilts such as VCP 21/121/221 contribute to overestimations observed for high MRMS echo tops. By comparison, strategies as VCP 12/212 with 14 elevation angles show fewer overestimations for equivalent MRMS echo-top heights. This is mainly due to precipitation echo tops located in the vertical gaps between successive elevation angles. For VCP 12/212, the role of the AVSET-controlled shortest VCP that cuts the highest tilts has also been investigated. AVSET increases the overestimations of echo tops higher than 6 km. The wider cone of silence causes overestimations when the echo top lies above the highest elevation scan. However, it seems that for low echo tops, the shorter refresh rate contributes to mitigating underestimations, especially in stratiform cases. The main explanations to the overestimations are the data voids and their treatment by the MRMS 3D interpolation scheme, which also suffer from spacing between 3D levels at high altitude. Beam broadening and the echo-top interpolation scheme also play a role.
To diminish the estimation errors in echo-top height, it is recommended to use the VCP 12/212 instead of VCPs with fewer tilts and greater voids between the highest tilts (VCP 21/121/221 was retired between 2018 and 2020; Kingfield and French 2022) in every precipitation case. Regarding the AVSET use, recommendations are either to diminish the 18 dBZ threshold or to use other strategies like Supplemental Adaptive Intravolume Low-Level Scan (SAILS) when forecasters need higher refresh rates at low levels. In MRMS, it is recommended to increase the number of levels above 9 km height to decrease overestimations, and to add a flag to echo tops when the interpolation cannot be performed because of missing data. Also, a postprocessed echo-top height product could be added in MRMS that accounts for the biases and uncertainties identified in this work. A prognostic error model for postprocessing will be developed as a follow-up study.
Future work includes a benchmarking of all echo tops products available, including from other radar networks, to quantify their behavior in different weather types and assess the strengths and weaknesses compared to the KuPR 18-dBZ echo-top height reference. A new intercomparison could be conducted to evaluate the effect of the new precipitation VCPs (112 and 215), both available since 2020.
Acknowledgments.
Author Mandement thanks François Bompay, Isabelle Beau, and the École Nationale de la Météorologie staff who made possible the internship leading to this work and Météo-France, which funded it. Funding for author Kirstetter was provided by the NASA Global Precipitation Measurement Ground Validation program under Grants NNX16AL23G and 80NSSC21K2045 and the Precipitation Measurement Missions program under Grant 80NSSC19K0681. Funding for author Reeves was provided by NOAA/Office of Oceanic and Atmospheric Research under NOAA–University of Oklahoma Cooperative Agreement NA21OAR4320204, U.S. Department of Commerce.
Data availability statement.
The computer code can be obtained on request from the corresponding author. Because of their proprietary nature, data created as a part of this research and the MRMS data used in this study cannot be made openly available.
REFERENCES
Amburn, S. A., and P. L. Wolf, 1997: VIL density as a hail indicator. Wea. Forecasting, 12, 473–478, https://doi.org/10.1175/1520-0434(1997)012<0473:VDAAHI>2.0.CO;2.
Awaka, J., M. Le, V. Chandrasekar, N. Yoshida, T. Higashiuwatoko, T. Kubota, and T. Iguchi, 2016: Rain type classification algorithm module for GPM Dual-Frequency Precipitation Radar. J. Atmos. Oceanic Technol., 33, 1887–1898, https://doi.org/10.1175/JTECH-D-16-0016.1.
Babin, S. M., 1995: A case study of subrefractive conditions at Wallops Island, Virginia. J. Appl. Meteor. Climatol., 34, 1028–1038, https://doi.org/10.1175/1520-0450(1995)034<1028:ACSOSC>2.0.CO;2.
Bureau of Transportation Statistics, 2015: Weather’s share of delayed flights national (January–December, 2015). Accessed 25 January 2023, https://transtats.bts.gov/OT_Delay/OT_DelayCause1.asp?20=E.
Cao, Q., Y. Hong, Y. Qi, Y. Wen, J. Zhang, J. J. Gourley, and L. Liao, 2013: Empirical conversion of the vertical profile of reflectivity from Ku-band to S-band frequency. J. Geophys. Res. Atmos., 118, 1814–1825, https://doi.org/10.1002/jgrd.50138.
Chrisman, J. N., 2009: Automated volume scan evaluation and termination (AVSET)—A simple technique to achieve faster volume scan updates. 34th Conf. on Radar Meteorology, Williamsburg, VA, Amer. Meteor. Soc., P4.4, https://ams.confex.com/ams/34Radar/webprogram/Paper155324.html.
Delobbe, L., and I. Holleman, 2006: Uncertainties in radar echo top heights used for hail detection. Meteor. Appl., 13, 361–374, https://doi.org/10.1017/S1350482706002374.
Evans, J. E., K. Carusone, M. M. Wolfson, M. Robinson, E. R. Ducot, and B. Crowe, 2004: Improving convective weather operations in highly congested airspace with the Corridor Integrated Weather System (CIWS). 11th Conf. on Aviation, Range, and Aerospace Meteorology, Hyannis, MA, Amer. Meteor. Soc., P1.5, https://ams.confex.com/ams/pdfpapers/81276.pdf.
Held, G., 1978: The probability of hail in relation to radar echo heights on the South African Highveld. J. Appl. Meteor. Climatol., 17, 755–762, https://doi.org/10.1175/1520-0450(1978)017<0755:TPOHIR>2.0.CO;2.
Hou, A. Y., and Coauthors, 2014: The Global Precipitation Measurement Mission. Bull. Amer. Meteor. Soc., 95, 701–722, https://doi.org/10.1175/BAMS-D-13-00164.1.
Howard, K. W., J. J. Gourley, and R. A. Maddox, 1997: Uncertainties in WSR-88D measurements and their impacts on monitoring life cycles. Wea. Forecasting, 12, 166–174, https://doi.org/10.1175/1520-0434(1997)012<0166:UIWMAT>2.0.CO;2.
Iguchi, T., and Coauthors, 2021: GPM/DPR level-2 algorithm theoretical basis doc. NASA Measures Algorithm Theoretical Basis Doc., 238 pp., https://gpm.nasa.gov/resources/documents/gpmdpr-level-2-algorithm-theoretical-basis-document-atbd.
Kingfield, D. M., and M. M. French, 2022: The influence of WSR-88D intra-volume scanning strategies on thunderstorm observations and warnings in the dual-polarization radar era: 2011–20. Wea. Forecasting, 37, 283–301, https://doi.org/10.1175/WAF-D-21-0127.1.
Kirstetter, P.-E., and Coauthors, 2012: Toward a framework for systematic error modeling of spaceborne precipitation radar with NOAA/NSSL ground radar–based national mosaic QPE. J. Hydrometeor., 13, 1285–1300, https://doi.org/10.1175/JHM-D-11-0139.1.
Kirstetter, P.-E., H. Andrieu, B. Boudevillain, and G. Delrieu, 2013: A physically based identification of vertical profiles of reflectivity from volume scan radar data. J. Appl. Meteor. Climatol., 52, 1645–1663, https://doi.org/10.1175/JAMC-D-12-0228.1.
Lakshmanan, V., K. Hondl, C. K. Potvin, and D. Preignitz, 2013: An improved method for estimating radar echo-top height. Wea. Forecasting, 28, 481–488, https://doi.org/10.1175/WAF-D-12-00084.1.
MacGorman, D. R., M. S. Elliott, and E. DiGangi, 2017: Electrical discharges in the overshooting tops of thunderstorms. J. Geophys. Res. Atmos., 122, 2929–2957, https://doi.org/10.1002/2016JD025933.
Maddox, R. A., D. S. Zaras, P. L. MacKeen, J. J. Gourley, R. Rabin, and K. W. Howard, 1999: Echo height measurements with the WSR-88D: Use of data from one versus two radars. Wea. Forecasting, 14, 455–460, https://doi.org/10.1175/1520-0434(1999)014<0455:EHMWTW>2.0.CO;2.
NOAA Virtual Lab, 2023: Products guide: xx dBZ Echo Top (ET). Accessed 25 January 2023, https://vlab.noaa.gov/web/wdtd/-/xx-dbz-echo-top-et-.
Office of the Federal Coordinator for Meteorological Services and Supporting Research, 2017: WSR-88D meteorological observations—Part C: WSR-88D products and algorithms. Federal Meteorological Handbook 11, FCM-H11C-2017, 396 pp., https://www.icams-portal.gov/resources/ofcm/fmh/FMH11/fmh11partC.pdf.
Schoor, T., 2016: NSSL technology helps the FAA. Accessed 25 January 2023, https://inside.nssl.noaa.gov/nsslnews/2016/06/nssl-technology-helps-the-faa/.
Smalley, D. J., B. J. Bennett, and M. L. Pawlak, 2003: New products for the NEXRAD ORPG to support FAA critical systems. 19th Conf. on Interactive Information Processing Systems, Long Beach, CA, Amer. Meteor. Soc., 14.12, https://ams.confex.com/ams/pdfpapers/57174.pdf.
Smalley, M., P.-E. Kirstetter, and T. L’Ecuyer, 2017: How frequent is precipitation over the contiguous United States? Perspectives from ground-based and spaceborne radars. J. Hydrometeor., 18, 1657–1672, https://doi.org/10.1175/JHM-D-16-0242.1.
Smith, T. M., and Coauthors, 2016: Multi-Radar Multi-Sensor (MRMS) severe weather and aviation products: Initial operating capabilities. Bull. Amer. Meteor. Soc., 97, 1617–1630, https://doi.org/10.1175/BAMS-D-14-00173.1.
Zhang, J., and Coauthors, 2011: National Mosaic and Multi-Sensor QPE (NMQ) system: Description, results, and future plans. Bull. Amer. Meteor. Soc., 92, 1321–1338, https://doi.org/10.1175/2011BAMS-D-11-00047.1.
Zhang, J., and Coauthors, 2016: Multi-Radar Multi-Sensor (MRMS) quantitative precipitation estimation: Initial operating capabilities. Bull. Amer. Meteor. Soc., 97, 621–638, https://doi.org/10.1175/BAMS-D-14-00174.1.
