a. Coverage

The three-way intercomparison study was conducted over the Arkansas–Red River basin, located in the southern plains. This location was chosen for several reasons. First, the NWS River Forecast Center (RFC) at Tulsa, Oklahoma, has been producing operational gauge-adjusted radar rainfall products for this watershed since 1993 and making these available to the research community. In addition, NEXRAD coverage in this area is extensive, with significant radar overlap [from two to four radar estimates depending on location; Maddox et al. (2002)]. Moreover, due to the relatively flat terrain in this region, terrain-induced artifacts and beam blockage are minimal. Figure 1 is a depiction of the Arkansas–Red River basin, showing both terrain elevation and the locations of NCDC surface rain gauges used for verification in this study. Note that spatial coverage of the gauges is fairly uniform with slightly higher density in Oklahoma.

Based on the availability of each of the three datasets, the period January 1998–June 1999 was chosen for study. During this period the P1 data were nearly complete, with only a few brief periods when the hourly estimates were not available, amounting to less than 1% of the total. The gauge data were generally available, but with occasional periods of missing data, amounting to less than 5% of the data. Nevertheless, there were enough surface gauge reports to provide a large statistical sample. The WSI precipitation dataset available for this study, which was obtained through Massachusetts Institute of Technology (MIT) Lincoln Laboratory, contains a larger number of data gaps. In our own experience collecting WSI data in real time over the last 2 yr, we find that the WSI data are always available, although our collection and archiving of the data sometimes fails. In any case Fig. 2 shows the percentage of the maximum possible 15-min data files present in our archive for each month of the 18-month study period. Overall WSI data were archived 77% of the time for 1998, and 81% of the time for the first 6 months of 1999.

b. P1 radar data

The hourly, 4-km-resolution rainfall maps produced at the ABRFC are an operational product that has been available since 1993. These rainfall estimates utilize both NEXRAD WSR-88D observations (Crum 1995; Fulton et al. 1998) and conventional surface rain gauge measurements. Details of the NEXRAD processing algorithms may be found in Fulton et al. (1998). Within P1 the NEXRAD reflectivity data are obtained from the hybrid scan, that is, the prescribed radar bins at a given azimuth–range that are closest to the ground, but remain largely unblocked, as discussed in O'Bannon (1997). These reflectivities are then mosaicked onto the Hydrologic Rainfall Analysis Project (HRAP) 4-km polar stereographic grid (Reed and Maidment 1999). Values for pixels with data from overlapping radars (points within 230 km of more than one NEXRAD site) are determined by choosing either the maximum or mean reflectivity, as determined by the operator. The initial values for rainfall are determined from the Z–R relationship in use at that time. In most cases (95%) the default relation (Z = 300R^1.4) is used. There is also the option of using relations corresponding to either tropical or cool season stratiform conditions. For NEXRAD sites operated by the NWS the Weather Forecast Office (WFO) determines the reflectivity–rainfall (Z–R) relation used. The minimum reflectivity considered to be raining is that corresponding to a rain rate of 0.1 mm h⁻¹ or roughly 11 dBZ, while maximum reflectivity is in the range of 51–55 dBZ, corresponding to 75–150 mm h⁻¹. The nationwide default maximum reflectivity is 53 dBZ, but individual sites may adjust this value depending on local environmental conditions (Fulton et al. 1998).

Initially, the processing algorithm followed a mean field bias correction approach known as Stage III, so-called because it corresponded to the third stage in the NWS processing of NEXRAD and gauge data, leading to the creation of a single gridded radar–gauge analysis over the entire CONUS (Stage IV). A description of the ABRFC Stage III algorithm (NWS 1997) may be found in Young et al. (2000). Since 1997 an updated version of the processing algorithm, known as P1 (process 1), has been implemented at the ABRFC (Young et al. 2000). The P1 algorithm uses local estimates of radar bias with respect to gauges to adjust the radar values toward the gauge observations based on a local bias correction. Mathematically, this bias correction consists of both a multiplicative bias and an additive bias (see Seo and Breidenbach 2002, appendix A). The multiplicative bias is applied for cases when the gauge to radar ratio is less than a predefined (tunable) threshold, while the additive bias is applied when the ratio exceeds this threshold. Note that the additive bias correction applies to the special case of a zero radar and a nonzero gauge estimate (thus avoiding an infinite multiplicative bias correction). The method essentially uses the radar data, where they are nonzero, to spatially interpolate gauge estimates, based on the idea that while the spatial variability or detail of the precipitation can be best captured by the higher-resolution radar, the gauges are better able to measure the absolute amounts of precipitation locally. Young et al. (2000) point out that the P1 algorithm performs best when gauge densities are high and bias ratios are relatively uniform. Operator intervention is used for manual quality control (QC) in a final step. In this step extensive effort is made to remove observed radar artifacts such as beam blockage and range effects. At this stage, the operator also has the capability to add “pseudogauges,” for example, in cases when there is reason to believe that rain or snow is not being detected by either the radar or rain gauge (B. Lawrence 2003, personal communication). Significantly, there is no record of which gauges were used in the analyses, and so it is not possible to precisely reconstruct or reproduce a given hourly analysis. The original effective precision of the estimates is 0.1 mm h⁻¹, although the data are saved at a precision of 0.01 mm h⁻¹ (Fulton et al. 1998) (see section 3c).

c. WSI radar data

The WSI radar data used in this study is the NOWrad SPECIAL precipitation product. This NEXRAD-based rainfall product is broadcast to subscribers via a satellite link in real time every 15 min. While the WSI processing algorithms are proprietary, general characteristics of the algorithms may be described here. The 15-min rainfall data are mosaicked over the CONUS on ∼2 km regular latitude–longitude grid. Starting with the original NEXRAD three-dimensional polar reflectivities, a hybrid scan is constructed by selecting from among the original tilt angle data. The removal of ground clutter, anomalous propagation, and other artifacts is done using a combination of automated techniques and real-time modification by trained meteorologists. (Again, precise details of how this hybrid scan is produced in the WSI processing are uncertain.) The mosaic is then created by taking the highest NEXRAD reflectivity value from overlapping radars (as discussed, points closer than 230 km to multiple NEXRAD sites) within each 2-km grid box. This is followed by a conversion from reflectivity to rain rate. The Z–R relations are based on a proprietary “weather condition” approach. In this approach the coefficients of the Z–R relationship are stored in a look-up table stratified by estimates of precipitable water and static stability from the National Centers for Environmental Prediction (NCEP) operational gridded analyses. These look-up tables are specified for each season and each NEXRAD site. While the Z–R relations employed have not changed in many years, the real-time QC operations are continually updated. Nevertheless, it is likely that the P1 algorithms are subject to a greater degree of QC than those of WSI and so we might expect a reflection of this in some of the verification results (section 3). This higher level of QC is related to the somewhat less stringent timeliness requirements of the P1 processing, which therefore allows more extensive manual quality control (Fulton et al. 1998; Young et al. 2000). The P1 processing strives to produce each new analysis hourly by no more than 35 min past that hour. Moreover, additional reprocessing of existing analyses is frequently done going back several hours based on arrival of 1200 UTC gauge data from the cooperative observer network. WSI analyses of 15-min accumulated rainfall over the entire CONUS on a 2-km grid are generally transmitted four times an hour within 10 min of the corresponding valid time.

During the study period the precision of the archived WSI data was 1.27 mm (0.05 in.) for the 15-min accumulated rainfall, meaning that this is also the smallest nonzero rainfall amount, which corresponds to a reflectivity of approximately 30 dBZ (assuming the default NEXRAD Z–R relation). The maximum 15-min accumulation allowed by this data format is approximately 20 mm. Since June 2000, the WSI data precision has increased to 0.254 mm (WSI 2000, personal communication), in part to account for very light rainfall amounts (see section 3c).

WSI data have been used by television and internet media sources (e.g., The Weather Channel), academic researchers, and commercial research and consulting firms. For example, Germann and Zawadski (2002) used these data in a study of a short-term precipitation forecasting method using pattern tracking. D. Curtis (2003, personal communication) has used WSI data in the development of gauge-adjusted radar rainfall estimates, with application to hydrologic prediction and regional climatology.

As noted, although the WSI data are distributed widely, and have been employed for both qualitative and quantitative applications by a variety of users, detailed intercomparison with rain gauges and existing NWS radar–rain gauge products have not been published. Evaluation of the WSI data should therefore prove useful to current or future users of this rainfall product.

d. NCDC gauge data

The rain gauge data are those available from the National Climatic Data Center (NCDC) or in repackaged form from other redistributors. The gauge set comprises those gauges located at NWS first-order reporting stations, as well as those belonging to other non-NWS organizations including federal, state, and local agencies, such as the Army Corps of Engineers, local water districts, etc. They do not include gauges from the Oklahoma Mesonet (Brock et al. 1995). Over the ABRFC basin, there were a total of 155 NCDC gauges used to assess the P1 and WSI analyses, although not all gauges reported at all times. Of the total, only a small number (less than 10%) were the newer tipping bucket gauges used as part of the Automated Surface Observing System (ASOS). These are generally heated and report hourly rainfall at higher precision of 0.254 mm (0.01 in.). The majority of gauges were either Fisher Porter or Universal unheated weighing rain gauges, which report hourly with a precision of 2.54 mm (0.1. in.) (see section 3c).

It should be noted that gauge measurements themselves also have known sources of error, which relate to gauge type, meteorological conditions such as wind and precipitation type (frozen or liquid), site characteristics such as topography and vegetation, as well as human error in recording and transmitting observations. Groisman and Legates (1994) and D. Curtis (2003, personal communication) summarize many error characteristics of U.S. gauge observations. For example, wind-induced undercatch can vary from 3% to 20%. Additionally, unheated gauges will often not record frozen precipitation until some hours or days after an event when melting occurs. NCDC subjects the gauge data to some quality control in an attempt to correct or remove suspect measurements. Although we did not observe any gross discrepancies in the gauge observations used here, it can be assumed that at least some of the variability seen in validation results must be due to gauge measurement errors.

e. Data preprocessing

In order to perform the intercomparison, the 15-min WSI data were first accumulated into hourly amounts. Any hour that contained at least one missing WSI 15-min file was assigned a missing value. Then, the WSI 2-km fields were simply averaged to a 4-km-resolution grid for compatibility with the spatial resolution of the P1 fields. The P1 data were also remapped from the HRAP grid to the 4-km WSI latitude–longitude grid using a nearest neighbor method. Collocation with gauges was done by choosing the single radar grid cell nearest to the latitude and longitude of the gauge.

In the sections below the results of the two-way and three-way intercomparisons for monthly, daily, and hourly rainfall accumulations are discussed in greater detail. As one progresses to shorter temporal scales, less agreement is expected between the datasets since time averaging tends to smooth random differences (or errors) in measurement. This effect is particularly important for rainfall events characterized by a high degree of temporal and spatial variability. Note that we use the terms “monthly” and “daily” broadly since, in the interest of maximizing sample sizes, data were retained even if individual hours may be missing for the time period in question. In all cases, all (two or three) data sources (i.e., P1, WSI, and gauges) are set to missing for any hour in which at least one of the data sources was missing in the comparison. This ensures that the statistics reported are based on identical samples.

Additional sources of disagreement between radar and gauge estimates are also possible simply due to so-called representativeness error: gauges measure precipitation falling at the surface at a single point integrated over time, while the radar is sensitive to hydrometeors present in a three-dimensional atmospheric volume, and over a much shorter period of time. In other words, even if both gauges and radars were “perfect” observing systems, one would expect differences due to sampling characteristics.

To gain additional insight into the potential impacts of any differences between the rainfall estimates on hydrologic forecasts, a comparison of storm event rainfall from P1, WSI, and rain gauge data for the Illinois River and its subcatchments was also performed for the months of January 1998, October 1998, and April 1999. The Illinois River and its subcatchments are shown later (Fig. 13) in detail with topography and rain gauge locations. Its location within the ABRFC basin is also highlighted, along with the locations of NEXRAD radars. Hourly rainfall from the radar products was aggregated for each study basin (Table 1) to produce hourly MAP values. Similarly, available rain gauges from the NCDC network (Table 2) were averaged to the grid points using inverse distance-squared weightings to create hourly gauge-derived MAP values. Missing data in either the radar or gauge rainfall products were excluded from the analysis.

a. Monthly precipitation

Monthly accumulated precipitation at each point on the 4-km grid was calculated by summing the individual daily totals for each day of the month. Figure 3 shows the estimated monthly precipitation for January 1998 for the two radar products over the Arkansas–Red River watershed. (In this and subsequent figures, the locations of the NEXRAD sites are indicated either by simple points or by the three-letter NEXRAD site identifier.) Precipitation accumulation during January 1998 ranged from 0 to 300 mm with the highest amounts observed in the southeast portion of the basin near the Oklahoma–Arkansas border. Total precipitation in the WSI and P1 estimates decreases sharply toward the west, with low amounts in the westernmost parts of the basin including southeastern Colorado and northeastern New Mexico. Of note is a “wedge” of reduced rainfall southwest of the Vance–Enid (VNX) radar; this is suggestive of beam blockage, a point we return to below (in the discussion of Fig. 6).

There appears to be good qualitative agreement between both monthly estimates, even though in the winter months frozen precipitation is likely with potentially adverse impacts on both radar and gauge estimates. Examination of hourly gauge reports for January 1998 reveal that there was some snowfall in eastern Oklahoma and Kansas, but the area affected was relatively small (seven gauges reported snow) and amounts were all between 5% and 12% of the monthly total liquid equivalent precipitation. For instance, Tulsa and Okemah reported the highest monthly snowfall of 102 and 127 mm, respectively—representing 11.5% of the monthly precipitation. Remaining gauges reported far less snowfall. For example, the Wichita WSO reported 15 mm, or 5.8%, of the monthly precipitation. Time series of the snowfall reports also indicate that the occurrence of snowfall was consistent with rainfall gauge reports at the same time, and that problems with delayed melting of snowfall were not significant. The monthly accumulation maps do not indicate any systematic differences that are specifically located in this region. Thus, we may infer that if P1 estimates were bias corrected toward the gauge totals in these areas, only a minor proportion of the differences between WSI and P1 totals during this month are due to errors in WSI estimates associated with snowfall.

The WSI total precipitation contained more small-scale variations than the P1 estimates (see, e.g., the rainfall in central Oklahoma). The P1 field is smoother in part because P1 processing applies an additive bias correction when gauge estimates are much larger than the radar estimates (see section 2b). This includes all cases of measurable gauge precipitation and zero radar precipitation. Under these circumstances the bias correction applied is essentially based on a gauge-only analysis that cannot resolve finescale features of the precipitation field (Seo and Breidenbach 2002) because of the typically large distance between gauges. The difference in smoothness is also consistent with the fact that, in grids with overlapping radar coverage, the WSI algorithms choose the maximum reflectivity, while P1 processing uses the mean reflectivity more than 90% of the time. Use of the mean would result in smoother rainfall analyses, since averaging tends to reduce random variations in the reflectivities.

The absolute and relative difference fields for January 1998 were also examined (not shown). In terms of absolute difference the largest discrepancies are seen in extreme southeast Oklahoma where the WSI totals are approximately 100 mm less than those of the P1 data. These differences correspond to relative differences of about 60%–80% of the P1 amounts. In addition, although not visible in these figures, enhanced images show several noticeable radar-based artifacts in the WSI data. The cause of these artifacts may be due to anomalous propagation, or to beam blockage, which may not be corrected for during generation of site-specific hybrid scans in the WSI processing. In fact, the pattern of differences implies that much of the underestimation is related to beam blockage and overshooting at the Little Rock site (SRX) near the Arkansas–Oklahoma border. Fulton et al. (1998) had discussed how the operational NEXRAD algorithms (and by extension P1) attempt to correct for these effects. The artifacts are shown more clearly in longer-term accumulations presented below (Figs. 5 and 6). Overall, the rainfall accumulation fields display horizontal patterns indicative of winter stratiform precipitation, which tends to be organized at larger spatial scales and contains relatively little small-scale organization.

In contrast to the winter precipitation, Fig. 4 shows results during August 1998. The overall accumulation fields show more small-scale variation. The different patterns of accumulation reflect the different nature of the summer convective precipitation compared to the winter precipitation that occur in this region. The precipitation for the month was much more uniformly distributed across the basin in August than in January. Totals are mainly in the range of 0–200 mm but in some small areas they approach 300 mm. The P1 accumulation field in August is also somewhat smoother than that produced by WSI, just as it was in January. Similarly, the totals in August are in good qualitative agreement in terms of the overall patterns, as they were in January. However, contrary to the January totals, WSI estimates for August were generally greater than those from P1. In some places these differences are greater than 100 mm. This may be due to both the mosaicking strategy employed in the WSI processing (i.e., using the maximimum reflectivity value in a 2-km grid cell), and contamination due to thunderstorm-induced hail, which might be less effectively screened in the WSI QC procedures. Unlike January, there are no obvious radar artifacts in the WSI totals. Similar results are seen in accumulated precipitation for the entire year (Fig. 5, left-hand panels). Both datasets produce annual accumulations in the range of approximately 100–1200 mm. Consistent with climatology of the region, the highest amounts are seen in the eastern part of the basin (Owenby et al. 1992). Interestingly, the geographic distribution of the differences (Fig. 5, right-hand panels) displays a coherent pattern: in the northeastern and southwestern portions of the basin, the WSI totals are somewhat larger than P1 (by approximately 100–200 mm), while in the southeastern and northwestern areas WSI totals tend to be less than P1, again by some 100–200 mm. In areas of disagreement, P1 totals were found to be closer to the NCDC gauge totals than those from WSI. Since there is no obvious correlation with elevation, or with known gauge characteristics or gauge density, and the pattern contains no linear or circular features associated with radar-related artifacts, we conclude that these differences are likely due to the Z–R relations employed in the WSI processing, which vary with season, location, and synoptic conditions (section 2c).

As noted in the results for January 1998, there are two distinct types of radar-based artifacts in the WSI data that are present to a much lesser extent in the P1 totals. Figure 6 shows rainfall accumulations for a subregion in eastern Oklahoma and extreme western Arkansas. The first type of artifacts likely associated with beam blockage are most evident immediately south and west of the Fort Smith, Arkansas, radar site (SRX), and due south of the Oklahoma City–Norman site (TLX). These are manifested in linear features of reduced rainfall extending radially from the radar sites. In the most extreme cases, the magnitudes of these reductions are on the order of 20%–70% of the P1 annual totals. While also present in the P1 analysis, the gauge–radar bias adjustment and/or improved QC has nearly eliminated this feature.

In addition, another set of radar artifacts are also detectable in the areas immediately surrounding some of the other radar sites, most notably at Tulsa–Inola (INX) and Vance–Enid (VNX). These areas of reduced accumulations in circular regions within 20–50 km from the radar can be on the order of 50% of the annual total estimated in P1, but are more generally in the range of 10%–15%. For example, the area immediately surrounding the Tulsa site is in excellent agreement with the P1 amounts, but at greater ranges totals are somewhat lower than P1. This phenomenon looks quite similar to that reported in the analyses of early NEXRAD data, which used an older algorithm to generate the hybrid reflectivity scan (e.g., Smith et al. 1996b). This was traced to the consistent use of higher tilt angle scans for close-in ranges in an attempt to effectively sample a constant height of 1 km above the radar. Implementation of a terrain-based hybrid scan algorithm (O'Bannon 1997) by the NWS around 1997 has alleviated this problem. While it was not possible to obtain the precise algorithm used to generate the WSI hybrid scans, it appears likely that it uses a strategy quite similar to the older NWS algorithms.

Results for the first 6 months of 1999 (not shown) mirror those seen for 1998. First, monthly totals show a similar geographic disribution to the 1998 data; namely, highest amounts in the east (between 800 and 1000 mm), with decreasing totals as one moves westward. Second, the difference fields are remarkably similar to those seen in 1998. As in 1998, WSI − P1 differences tend to be positive in the northeast (greater than 50 mm) and southwest. Negative differences (also greater than 50 mm) dominate in a zone stretching from the southeast to the northwest. Note that these patterns extend over more than one annual cycle. As in 1998, there are artifacts in the WSI data that are not seen in the P1 totals. Evidently, the WSI quality control procedures are not completely effective in removing some of these artifacts. This is not unexpected since P1 products are subject to a higher level of quality control (section 2c).

Finally, a more compact depiction of the two datasets in shown in scatter diagrams of monthly totals for the periods January–December 1998 and January–June 1999 in Fig. 7. A point is plotted for each month and for each grid location within the Arkansas–Red River basin. These scatter diagrams show that despite the presence of radar-based artifacts, and some noticeable differences at individual points, the overall agreement between the WSI and P1 totals is rather good. The mean difference between the two datasets is negligible (absolute value less than 4 mm), the aggregate rms difference (mean difference removed) between WSI and P1 monthly total rainfall was approximately 25 mm, and the correlation coefficient was greater than 0.85 in both years. Thus, for developing longer-term climatologies, the WSI and P1 data are roughly equivalent, although some care must be exercised since site-specific radar artifacts have not been completely removed from the WSI estimates, and because the WSI analyses contain more small-scale variability that those of P1.

b. Daily precipitation

Progressing to shorter temporal scales, the daily accumulated rainfall data were also analyzed. For this analysis NCDC rain gauge observations were used and therefore the comparisons are limited to the locations of the gauges. We have excluded all points where either the gauge, WSI, or P1 reported zero rainfall for the 24-h period, to focus on cases when rainfall was occurring. This data selection procedure reduces the correlation-enhancing effect of the many cases where all estimates are zero.

Figure 8 shows scatter diagrams for WSI versus rain gauge, P1 versus rain gauge, and WSI versus P1 daily rainfall during January–March 1998. Similar analyses were performed for all other months in the 18-month study period (not shown). Additionally, daily accumulation difference statistics (i.e., mean and rms difference, and correlation of daily rainfall) as a function of month were examined. Figure 9 shows the difference statistics binned by month, for 1998 and 1999. The top panel also shows the sample size used in each month. Since the daily statistics were calculated by aggregating all collocation points (155 per day) over all days of the month, any variations in spatial variance only, would be effectively masked (i.e., the statistics represent a spatial and temporal aggregation).

The scatterplots show that both WSI and P1 daily amounts agree fairly well with surface gauges, but that in terms of correlation coefficient and rms differences the P1 estimates are closer to the gauge values. This is expected since the P1 processing makes explicit use of gauge observations. In fact, the use of gauges in the P1 data makes any independent validation of the two radar-based estimates quite difficult. Available P1 data do not specify which gauge observations were actually used in the processing, nor does the ABRFC retain this information. For this reason we must assume that some information from the gauge dataset is present in the P1 radar estimates. A more independent validation might use gauge data from the Oklahoma Mesonet (Brock et al. 1995). However, even these gauge observations are often used in P1 processing (B. Lawrence 2003, personal communication).

Based on the scatterplots and difference statistics, several characteristics are noted. First, the mean difference of the WSI daily data with respect to gauges tends to be negative during the cold season, and positive in the warm season. The magnitude of these differences ranges between ± 3–5 mm day⁻¹ for rainy days. This pattern is present throughout the study period. Second, correlation of the P1 estimates with gauges is higher in winter than summer. During winter, the correlations are noticeably higher than those of the WSI data, while during summer they are comparable. Correlations of the two radar estimates with each other are generally greater than 0.7.

There are several possible reasons for these differences. First, WSI underestimation during the cold season could be due to the radar beam overshooting the average hydrometeor levels during stratiform rainfall events, which are characterized by low cloud-top heights (Smith et al. 1996b; Kitchen and Jackson 1993). In addition, radar reflectivities tend to be lower for pure frozen precipitation (snow) and if Z–R relationships are not adjusted accordingly, significant underestimation is likely. On the other hand, uncorrected brightband contamination due to melting snow would lead to overestimation—contrary to what was observed. Even if winter and summer hydrometeors were present at the same levels, differences in precipitation type and particle size distributions would lead to different vertical profiles of reflectivity. In either case, P1 estimates, being calibrated against local gauge measurements, are much less likely to exhibit these tendencies. Second, summer precipitation events in this region are characterized by frequent convective outbreaks, which may contain significant amounts of mixed precipitation (i.e., hail, graupel, ice falling through melting layers, etc.), which might also induce anomalously high reflectivity and spuriously high rain rates. If quality control procedures do not screen for these effects, overestimation is likely. Again, P1 estimates would tend to be recalibrated using nearby gauges, assuming they were available and able to observe these small-scale phenomena. Additionally, by design the WSI algorithms consistently take the largest of all pixel reflectivity values occurring at points observed by multiple radars (WSI 2000, personal communication). There may be a tendency for overestimation at these points.

The mean difference of the P1 data with respect to the gauges is quite small throughout the year. As noted, this is expected based on the P1 algorithms, which are explicitly designed to perform a bias correction based on local gauge − radar differences. Rms differences with respect to gauges are comparable for both radar estimates, ranging between 5 and 15 mm day⁻¹, with the WSI estimates exhibiting a slightly higher value (approximately 2–3 mm day⁻¹) than the P1 data. The only exception is a somewhat anomalous spike that is seen during September and October of 1998 in which the rms deviations reach 20–25 mm day⁻¹. At the same time, the corresponding mean differences and correlations for these months are unexceptional. A possible explanation for this behavior is that, as seen in additional statistical measures (not shown), this is a period of greater than normal rainfall and rainfall variability, which would tend to result in larger rms differences. And while the correlations show no obvious seasonal trend, the sharp drop in correlations with respect to gauges for April 1998 may be an artifact of the reduced sample size in that month.

Across most of the statistical metrics, the P1 and the gauge data appear quite similar, indicating that the processing algorithms implemented at ABRFC have successfully produced a radar–gauge product with many of the same aggregate statistical qualities; this is one of the goals of the P1 algorithms. Nevertheless, closer examination of the data themselves, rather than the aggregate statistics, has also shown some small, but quantifiable, inaccuracies in the P1 estimates. These will be discussed in section 3c.

As noted, the WSI daily estimates show somewhat less agreement with gauge observations throughout the 18-month period. For example, WSI rms differences are 1–2 mm day⁻¹ higher than those of P1. Also, WSI mean differences range between −5 and 10 mm day⁻¹, with correlations generally between 0.4 and 0.7. In comparison, P1 mean differences are close to 0, and correlations are generally between 0.6 and 0.8.

c. Hourly precipitation

Hourly precipitation estimates were evaluated for the entire 18-month period. As described in section 2e, only gauge locations and hours in which all three data sources had nonmissing values were considered in the analysis.

Figure 10 contains the cumulative probability distribution functions (CPDFs) of estimated hourly rainfall for June–August 1998. The most obvious feature is the effect of the data precision (or quantization) on the CPDF curves. With a precision of 0.01 mm h⁻¹ the P1 curves are a more or less continuous function of rain rate. The CPDFs for the WSI estimates, with an effective quantization of 1.27/4 mm h⁻¹, have obvious discontinuities, but the effect is most obvious in the rain gauge CPDFs, with clear jumps at intervals of 2.54 mm h⁻¹, the reporting precision of most of the gauges in the dataset. The division by 4 for the WSI precision occurs because we average four 2-km estimates for each 4-km estimate. Nonzero values for gauge amounts less than 2.54 mm h⁻¹ indicate that an extremely small fraction of gauges were reporting at a higher precision. Despite the differences in precision and the fact that the data sources are independent, the WSI and gauge CPDFs are quite similar for rain rates above 2.54 mm h⁻¹.

One of the largest differences between the WSI and P1 CPDFs is the much higher proportion of low rain rates in the P1 data. For example, during June 1998, nearly 80% of the P1 estimates were less than 2.5 mm h⁻¹, while for the WSI data this proportion is less than 40%. This result is consistent through all the data, with only small variations from month of month. In absolute terms, the P1 data contain more than three times the number of occurrences of rainfall greater than 0 and less than or equal to 1 mm h⁻¹ than the WSI data. Unfortunately, the low precision of gauge data (2.54 mm h⁻¹, or 0.1 in.) means that there are few observations until the 2–3 mm h⁻¹ threshold is reached. Since very light rainfall events (less than 2.54 mm h⁻¹) would likely be reported as 0, these data cannot be used as an independent verification in these situations. In all three datasets there is a tendency for the percentage of lower rain rates to drop during summer (e.g., the proportion of rain rates less than 5 mm h⁻¹ is approximately 10% higher in winter than in summer). This reflects the increase in convective precipitation in summer, which typically contains higher rain rates than winter stratiform storm systems.

The hourly estimates were also analyzed in the form of multicategory contingency tables. Cooccurrences of rain rates were tallied within each of several rain rate bins, or categories. The resulting figures (Fig. 11) are equivalent to binned scatterplots for the hourly rain rates. The rain-rate categories were chosen to be nonuniform, with smaller bins at low rain rates and larger bins for higher rainfall amounts. A separate table was constructed for each month and dataset pairing (e.g., WSI versus gauge estimates during January 1998, etc.). Zero precipitation values are allowed, but the (0,0) category is not included in the figures. Here, results for January and June of 1998 are shown, which are typical of the whole period.

Both the WSI–gauge and the P1–gauge comparisons show that there are a significant number of undetected rain events where the gauge recorded measurable rainfall while the radar-based estimates did not, and vice versa. The P1–gauge tables also show the large occurrence of very low rain amounts for gauge observations of no rain (the left column in each table). This tendency is even more pronounced in the WSI–P1 comparisons where, for WSI observations of 0 mm h⁻¹ (the bottom row in each table), there is frequently a peak in P1 occurrence at 0.2–0.5 mm h⁻¹. Aside from this discrepancy, the WSI and P1 hourly data show better overall agreement with each other than they do with gauges, in that more of the cooccurrences are present along the main diagonal, and in the sense that they are equivalent at delineating rain–no-rain events for rainfall events above 2 mm h⁻¹.

The large number of events when P1 detects rain but WSI does not may be due to a number of causes. First, light rainfall may be underdetected (relative to gauges) by either the NEXRAD system, or the WSI processing algorithms. For example, wintertime low-level stratiform precipitation may be completely undetected by even the lowest-elevation scans. Alternatively, while the P1 data are, by definition, correct at the gauge locations themselves, interpolation of gauge information to points far from the gauges may result in some overestimation of the areas covered by very light precipitation. Finally, the lower occurrence may be a result of the coarser discretization of this dataset, since some precipitation events with true 15-min rain rates less than 0.635 mm (i.e., 1.27/2) over widespread areas would inevitably be reported as 0 in the WSI data.

Figure 12 shows a pair of hourly images valid at 0500 UTC 16 February 1998. The data show two significant precipitation events, one occurring over north Texas, and the other over Arkansas. Also plotted are rain gauge locations for which the corresponding gauges recorded rainfall that was not observed in the WSI data. (Note that there may have been additional gauge observations used in the P1 processing that were not present in the NCDC data used here.) While both datasets accurately depict the locations of highest rain rates, which are in the range 3–8 mm h⁻¹, the P1 data also contain wide swaths with very low rain rates less than 1 mm h⁻¹. For example, see the area extending northwest from Texas into Colorado. In addition the core rain areas with higher rainfall rates are occasionally smoother and have reduced intensity in the P1 data, as in the precipitation center over north Texas.

It appears likely that the WSI data are sometimes missing lighter precipitation for the reasons described above (e.g., rainfall over Arkansas), but it is also likely that some of the spatial patterns of the rainfall present in the P1 data are somewhat unrealistic. These characteristics were observed throughout the P1 dataset during the entire 18-month period. This visual comparison is borne out in the previous analyses presented in this section (Figs. 10 and 11). A likely interpretation is that in the cold season it will be common for low-level light stratiform precipitation to fall undetected, completely underneath the radar beam. In areas with sparse gauge coverage the P1 algorithm will do a spatial interpolation of gauge–radar biases between gauges, since in fact the gauges are the only other source of data.

In sum, both WSI and P1 analyses appear to depict reasonable precipitation patterns where significant rainfall occurs. However, WSI analyses will not depict spatially, nor quantitatively, very light precipitation events. The P1 analyses typically indicate the occurrence of light precipitation events well, but appear to spread light rainfall amounts into somewhat unrealistic patterns.

The P1 algorithms emphasize adjustments to the radar estimates based on a bias correction procedure using nearby gauges. This procedure will, by definition, produce better agreement with gauge data as defined by the aggregate statistical metrics; as a biproduct, hourly maps will sometimes reveal light rainfall artifacts. Importantly, the presence of these light rainfall observations may not be critical for most applications, for example, in building unbiased climatologies. Moreover, a primary use of the P1 analyses at the ABRFC is to serve as input to hydrologic models, resulting in forecasts of river flow within the basin. These predictions are unlikely to be affected by the low rainfall rates, although detailed hydrologic simulations would be needed to confirm this.

d. Catchment storm analysis

For hydrometeorological applications, such as severe weather warning and flood and flash flood forecasting, the time and space scales of interest are typically limited to storm events lasting from a few hours to several days over basins of moderate to large scale (1000–10 000 km²). Hydrologic models operating at these time and space scales are sufficiently advanced to reproduce the primary runoff and routing mechanisms occurring during storm and interstorm periods. Due to the high sensitivity of both lumped and distributed watershed models to rainfall forcing, a catchment-scale intercomparison of storm events was performed; P1, WSI, and rain gauge products are used to highlight potential impacts of rainfall errors on hydrologic applications.

A limited sequence of rainfall events were chosen from the 18-month period that were representative of different seasonal characteristics identified during the monthly comparisons. Storm events in January and October 1998 represent the fall and winter stratiform precipitation type illustrated in Fig. 3, while storm events in April 1999 represent the convective storm regime typical of spring and summer (Fig. 4). Although a limited sample size, the chosen storms represent the predominant types of hydrometeorological conditions in the study area (Bradley and Smith 1994). For each storm event, estimates of the mean areal precipitation (MAP), fractional coverage at a specified rainfall threshold, and total accumulated rainfall over the period provide metrics for comparing the P1, WSI, and gauge-derived hourly rainfall maps.

The Illinois River basin was chosen to carry out the catchment-based analysis due to its reliable NEXRAD coverage over the study period. Prior comparisons of radar and rain gauge precipitation products (e.g., Johnson et al. 1999) have also focused on the Illinois River and its five subcatchments (Fig. 13 and Table 1). In addition, the performance of lumped and distributed hydrologic models for the Illinois River using NEXRAD rainfall data has been assessed by the National Weather Service (Finnerty et al. 1997; Smith et al. 1999). The scale of the Illinois basin (see Table 1) is typical of the 143 hydrologic units that constitute the ABRFC forecast area and ensures a large sample size of radar pixels falls within the basin (260 pixels for the 4-km product).

For each storm event and basin, the hourly MAP was computed. Figure 14 illustrates the three-way intercomparison for the Illinois River basin during the (a) 4–8 January 1998 and (b) 23–26 April 1999 rainfall events. Basin-averaged rainfall intensities are comparable among the three products. We note, however, that the discrepancy in the peak storm intensity (e.g., 14 mm h⁻¹ for P1 versus 9 mm h⁻¹ for WSI for 4 January 1998) observed in the MAP product will invariably lead to marked differences in observed and modeled hydrologic response, in particular for threshold-based infiltration schemes. For both storm events, the P1 MAP product more closely resembles the gauge-derived MAPs than does that of WSI, presumably due to the use of gauges for local bias correction of P1 data. The sole exception is the event occurring on 23 April, during which both P1 and WSI MAP estimates are significantly greater than the gauge MAP. This is likely due to the presence of small-scale precipitation cells that were not well sampled by the relatively sparse gauge network (see Fig. 13), leading to an underestimate of MAP during the period.

The discrepancy in the estimation of low rainfall amounts depicted visually in Fig. 12 is further quantified here by deriving the fractional storm coverage at specific rainfall intensities for the selected basins. Fractional coverage is simply the proportion of the basin covered by rainfall at or above a given rainfall intensity. Fractional coverage was computed at various threshold rain rates. Here, we illustrate results for the Illinois River at low rainfall rates (0 ≤ r ≤ 2 mm h⁻¹, fractional coverage fc₂) and high rainfall rates (r > 10 mm h⁻¹, fractional coverage fc₁₀), where r is the rainfall amount in the 4-km grid box (Fig. 15). In Figs. 15a and 15b, P1 data, but not WSI data, are characterized by high fractional coverage of the watershed of low rainfall amounts (fc₂) throughout the time period of the January 1998 storm. This is consistent with the maps shown in Fig. 12. As discussed previously, this is a likely result related to interpolation of sparse gauges (reporting nonzero rainfall) in the P1 estimates, as discussed earlier. This effect may be seasonally dependent, as the April storm event does not exhibit this tendency. For a high rainfall threshold, the two products have similar fractional coverages (fc₁₀), suggesting that the differences are limited to the estimation of low rainfall. In terms of hydrologic applications, actual model simulations would be required to determine whether or not the occurrence of high fractional coverage at low rain rates leads to errors in the estimation of antecendent soil moisture conditions or interstrom water balance.

Finally, the bias between the P1 and WSI data was further explored by comparing the monthly total basin-averaged rainfall for all study basins and storms. This metric captures the amount of total rainfall incident on a basin during each storm event. For a hydrologic model, the storm rainfall volume dictates the amount of runoff to be produced by various mechanisms (e.g., infiltration excess, saturation excess, baseflow). It is a key quantity affecting the runoff volume and hence the flood forecast. Figure 16 illustrates that P1 and WSI estimates of storm volume differ substantially for the tested basins and storms. Seasonality is observed in this bias, as the WSI is greater than P1 during April 1999 (spring–summer convective season) and less than P1 for January and October 1998 (fall–winter stratiform season). There is some difference between basins due to the spatial variability in rainfall over the domain. This can also be seen in Fig. 17, which shows that the January 1998 and April 1999 storm total products for P1 and WSI differ in terms of the smoothness of the rainfall field and location of maximum (normalized) rainfall intensities. These fields also suggest that the WSI storm totals contain somewhat more small-scale variability relative to the P1 totals, which is consistent with features seen in individual hourly images. These difference in the two-dimensional distribution of observed storm total rainfall could affect the hydrologic predictions from models that are sensitive to spatially variable precipitation forcing.

Although somewhat beyond the scope of this study, we note that Grassotti et al. (2002b) and Vivoni et al. (2001) have conducted hydrologic simulations over the Baron Fork in northeastern Oklahoma using a distributed, variable-resolution hydrologic model forced with both WSI and P1 rainfall estimates. For one storm event in October 1998 they showed that MAP differences of 6 mm h⁻¹ between the two products resulted in a change in predicted peak streamflow at the basin outlet of 175 m³ s⁻¹ (i.e., P1 and WSI peak rain rates of 20 and 14 mm h⁻¹ resulted in streamflows of 250 and 75 m³ s⁻¹, respectively). This sensitivity indicates that some care must be exercised when calibrating and running hydrologic models with radar-derived rainfall rates.