1. Introduction

A trade-off exists in warning for tornadoes between the probability of detection (POD) and false alarms. In the limit, false alarms could be avoided by never issuing warnings, whereas warning for every thunderstorm will increase the POD to 100%. The National Weather Service (NWS) uses the POD and the false-alarm ratio (FAR) as metrics for tornado warning performance, reflecting the intuition that both measures affect the public’s response to warnings. Recently Simmons and Sutter (2008) showed that tornado warnings with lead times up to about 17 min reduce tornado fatalities and injuries, but to date no evidence exists on false alarms and casualties.

We examine the influence of false alarms on casualties using tornadoes in the contiguous United States from 1986 to 2004. Our unit of observation is the individual tornado, but false alarms are nonevents. How precisely false alarms will affect tornado casualties is unclear. We hypothesize that residents will use the recent local FAR in assessing warning credibility, and construct multiple FARs based on different geographies and time horizons. We then include the FAR variable in a regression analysis of tornado casualties. We do not examine the link between false alarms and warning response directly, but since warning response affects casualties and our 19 yr of data include over 21 000 state tornado segments, we seek to detect an impact on response through casualties.

We find a strong relationship between the FAR and both tornado fatalities and injuries. We define FARs based on three different geographies [states, NWS Weather Forecast Offices (WFO), and television markets] and two time horizons (the past year and past two years). The results are remarkably consistent across the various definitions of FAR: a one-standard-deviation (1 std dev) increase in the FAR increases expected fatalities by between 12% and 29% and expected injuries by between 14% and 32%. The reduction in the national FAR between 1986 and 2004, although modest, reduced expected fatalities by 4%–11% and expected injuries by 4%–13%, based on the definition of FAR. We also evaluate the trade-off between FAR and POD depicted by Brooks (2004) on casualties. The point estimates of the impact of warnings and false alarms on casualties nearly offset each other, so we find no evidence that the NWS could reduce casualties by trading off a higher POD for higher FAR, or vice versa.

This paper is organized as follows: The next section discusses the intuition behind the false-alarm or cry-wolf effect, as well as evidence on false alarms and warning response to different natural hazards. Defining false-alarm variables for use in an analysis of casualties from tornadoes is challenging because false alarms are nonevents. Section 3 defines our false-alarm ratios and present preliminary evidence on the influence of false alarms on casualties. Section 4 defines the other variables employed in the analysis and discusses the econometric specification, and section 5 presents the regression results. Section 6 considers two extensions of our analysis, to consider the decline in tornado casualties attributable to the reduction in the false-alarm ratio since 1986, and the casualty trade-off between the probability of detection and false alarms in warning for tornadoes. The final section contains conclusions.

2. Theory of response to tornado warnings and the role of false alarms

The value of a weather forecast or warning depends on the accuracy of the forecast. A tornado warning is a binary forecast, and so a 2 × 2 matrix can be applied to attempt to estimate the value of the forecast (Katz and Murphy 1997). The error probabilities of the warning—the probability that a tornado occurs without a warning being issued (a missed event), and the probability that a tornado does not occur when a warning is issued (a false alarm)—drive the value of the warning. The first error probability is 1 − POD, and the second error is the FAR.¹ The expected-utility approach predicts that people will respond to warnings they receive when the net expected utility of response is positive. An increase in either error probability reduces the value of a warning. The net expected value depends on the losses avoided by response (the reduction in likelihood of being killed or injured) and the cost of response (interrupting activities to take cover) in addition to the error probabilities. If the FAR is sufficiently high, the net value of responding to a warning becomes negative.

Intuition and theory suggest that false alarms should reduce warning response, yet evidence on this in field studies is sparse. Breznitz (1984, p. 11) defines the false-alarm effect as the “credibility loss due to a false alarm” and describes the effect as follows: “Each false alarm reduces the credibility of a warning system. The credibility loss following a false alarm episode has serious ramifications to behavior in a variety of response channels.” In the framework of the value of forecasts and warnings, the effect occurs when residents judge the FAR to be sufficiently high that warnings are ignored. Evidence of a false-alarm effect in field studies, however, is sparse. Breznitz (1984), for example, provides evidence consistent with the effect, but from a laboratory setting, not for warning responses in the field. Field research found a reduction in credibility for earthquake predictions after a quake predicted by I. Browning on the New Madrid fault for December 1990 failed to occur (Atwood and Major 1998). The relevance of this evidence is questionable, because the warning was issued by a private individual instead of a government agency, and there was no scientific basis for such a targeted prediction. Field evidence against a false-alarm effect is provided by Dow and Cutter’s (1998) study of hurricane evacuation. They found no evidence of a lower evacuation rate in South Carolina for Hurricane Fran in 1996, which occurred just weeks after a false-alarm evacuation for Hurricane Bertha, and conclude, “The influence of ‘premature evacuations’ for Bertha played only a minor role in evacuation decisions for Hurricane Fran in South Carolina” (p. 249). Recently Barnes et al. (2007) observed, “Evidence for the cry-wolf effect in natural hazards research, however, has not been forthcoming.” Based on this, researchers may not be surprised if false alarms did not increase tornado casualties.

We test for a false-alarm effect for tornado warnings using NWS warning verification records and casualties from over 20 000 tornadoes across the contiguous United States. We will use the actual warning performance, or the objective FAR as calculated from NWS warning records, in our analysis. But residents’ warning response in the expected utility model depends on their subjective perception of warning quality, that is, what they perceive the FAR to be and not the objective FAR. Subjective perceptions may diverge from objective measures, yet researchers do not have access to the subjective estimates of warning quality for the general population. Our approach is valid if subjective perceptions are distributed around the objective probability, with some people overestimating and others underestimating the FAR. Furthermore, our approach really only requires that differences in the objective FAR affect casualties, and not that the average of residents’ subjective perceptions of FAR be equal to the objective FAR.

Another limitation with NWS warning verification records is the potential that the warning performance measures may not correlate closely with residents’ perceptions of hazard events. Barnes et al. (2007) discuss several such potential problems and argue that an event that is technically a false alarm in the NWS warning statistics may be regarded by residents as a near miss and thus a credible warning. For instance, false alarms can result from a storm that generated a funnel cloud that did not quite touch down, or a tornado may have occurred just outside of the warning area or just after expiration of the warning. The issuance of tornado warnings based on counties until October 2007 could also artificially inflate the NWS FAR. For example, if four counties were warned for a supercell thunderstorm that produced a strong, dangerous tornado in only one county, the official FAR for this event would be 0.75, approximately equal to the national average FAR, even though residents may consider this to be a well-warned event and increase their subjective evaluation of warning reliability as a consequence. Barnes et al. suggest that such close calls and near misses may not reduce the public’s confidence in tornado warnings. Thus a poor correlation between what officially counts as a false alarm and subjective evaluation could also explain a lack of an impact of false alarms on casualties.

We test for an impact of false alarms on tornado casualties, not warning response. Warning response affects casualties but is not the only determinant of casualties. We use a dataset with numerous control variables, because false alarms (or warnings) will only increase (decrease) casualties if all other factors affecting casualties are held constant. The control variables include the Fujita-scale (F scale) rating of the storm, time of day, month, and storm-path characteristics. The use of a large dataset of over 20 000 tornadoes should help to smooth out any variation in warning response. Our national dataset allows us to exploit considerable differences in FAR to detect a possible influence of false alarms.

3. False-alarm variable definitions

Theory tells us that warning performance should affect subjective estimates of warning quality, but theory does not inform us of exactly how residents form their perceptions. Also, our data points are tornado events. The warning issued for a tornado (if any) thus can be naturally included as a control variable for the individual tornado. But false alarms are nonevents and cannot be matched with specific tornadoes. We hypothesize that residents will use recent, local warnings to try to estimate the FAR. Residents need to use warnings over some period of time to estimate the true FAR yet distant events fade from memory and warning performance varies over time, most notably with the introduction of Doppler weather radar (Simmons and Sutter 2005). Residents should use local warnings as well, meaning warnings that they hear or see, to estimate the quality of warnings. Residents of Kansas are unlikely to use warnings issued for New York, Florida, or California to evaluate the reliability of warnings.

Because what constitutes local and recent is subject to differing interpretations, we construct FARs using three geographies and two time horizons to ensure the robustness of any findings. The geographies are 1) states, 2) NWS WFO County Warning Areas (CWAs), based on the CWAs of the WFOs of the modernized NWS, and 3) television markets as defined by the A. C. Nielsen Company.² We apply 1- and 2-yr lags in constructing FARs for each geography and refer to these as 1- and 2-yr state, WFO, and TV FARs. If a tornado occurs in, say, May 2004, the 1-yr window would be all warnings in the relevant geographic area in the months of May 2003–April 2004 and the 2-yr window would be May 2002–April 2004. All tornadoes in the same geography in the same month have the same FAR.

We do not expect that residents actually know the FAR for their state or TV market over the period. Our approach should not be interpreted too literally. We hope that residents’ subjective perceptions of warning quality correlate with our objective measures. That is, residents of a state with a 1-yr FAR of 0.95 should be more likely to think that “warnings are always false alarms” than residents of a state with a 1-yr FAR of 0.25. Our approach relies on these impressions correlating with actual recent local warning performance. If more residents take warnings seriously where recent warning performance has been good, the improved warning response should reduce casualties, and our regression models should detect the casualty reduction.

A complication arises when tornadoes occur in a geography where no warnings have been issued in the previous year or two. The FAR is undefined in this case, and it is unclear how residents will treat these warnings. Most of these tornadoes occur in states with low tornado frequency. The number of tornadoes with no warnings in the area over the period is 332 for 1-yr state, 770 for 1-yr WFO, and 1215 for 1-yr TV markets. Enough tornadoes occur with no warnings for most of the definitions that we code the FAR equal to 1 for these tornadoes and include a dummy variable “no warnings” as a control. This allows us to include these tornadoes without restricting how these warnings affect casualties. Too few tornadoes occur with no warnings in the state over the previous 2 yr to control for with a dummy variable. As a result, the regressions omit these tornadoes.

Our empirical design seeks to exploit variation in the FAR across locations and time to test for a false-alarm effect. This design requires sufficient variation in FARs to affect residents’ perceptions of warning quality and response to warnings. The summary statistics exhibit substantial variation, with 1-yr FARs ranging from 0 to 1, and 2-yr FARs ranging from under 0.5 to 1 (Table 1). The mean and median FAR are about 0.75 by each definition. The standard deviations are slightly larger with the 1-yr than 2-yr FARs, as a longer window of observation allows a more precise estimate of the FAR. The distribution of tornadoes across intervals for each of the six FAR definitions confirms the variation (Table 2). For the 1-yr FARs, between 9.5% and 21.5% of tornadoes occur with a FAR in excess of 0.9 while between 7% and 17% have FARs less than 0.6. Considerable variation exists within states as well; 1-yr FARs for most states exhibit a range of 0.5 or greater for most states. Figure 1 displays the state 1-yr FAR for Kansas over the period, which is typical of tornado-prone states, with variation from 0.5 to 0.9. Considerable variation in warning performance also exists across the country. Figure 2 displays a scatterplot of the FAR for the period of analysis for each of the WFOs in our dataset, along with the POD as calculated from our dataset (i.e., state tornadoes as reported in the Storm Prediction Center archive as opposed to county tornadoes). The FAR for all warnings over the period ranges from just over 0.5 to 1.0 for one WFO while the POD ranges from less than 0.1 to 0.8. The distribution of types of tornado outbreaks varies across the country (Trapp et al. 2005); to provide perspective on how this affects the variation in our data, we also report the aggregate FAR and POD over the period by state, for states with at least 100 tornadoes over the period (Table 3).

Table 4 reports preliminary evidence on the relationship between FARs and casualties. The table presents fatalities and injuries per tornado for each of the intervals from Table 2 and reveals a positive relationship between FARs and casualties. Table 3 reports the statistics for the 1-yr FAR definitions, and the pattern is similar for 2-yr definitions. The highest fatalities per tornado and highest or second highest injuries per tornado occur in the 0.9–0.999 interval. The intervals for 0.8 or greater feature the three highest fatality rates for state 1 yr and TV 1 yr, and three of the four highest rates for WFO 1 yr. The difference in rates is also pronounced, with the casualties per tornado for the smallest FAR intervals generally less than half of that for the 0.9–0.999 interval. The relationship across intervals is not monotonic, and casualties per tornado do not control for F-scale ratings or tornado or path characteristics. Nonetheless, this first look at the data suggests the existence of a false-alarm effect.

4. Control variables and the regression model

Tornado casualties take on nonnegative integer values and are count data. Ordinary least squares (OLS) estimation does not account for the censoring of the dependent variable at zero with count data (i.e., that casualties cannot take on negative values). Economists typically employ a Poisson regression model for analysis of count data of this type. The Poisson regression model specifies that the dependent variable y_i, in this application the number of persons killed or injured in a state tornado, is drawn from a Poisson distribution (Greene 2000, 880–886). The probability of a given number of casualties is

View Expanded

The parameter λ_i of the distribution is assumed to be a log-linear function of the independent variables x_i:

View Expanded

The Poisson model assumes equality of the conditional mean and variance of the dependent variable, and violation of this assumption is known as either overdispersion or underdispersion. The negative binomial generalization of the Poisson model is employed when count data exhibit overdispersion (Greene 2000, 886–888). The negative binomial regression model generalizes the Poisson model by introducing an individual, unobserved effect into the conditional mean, so

View Expanded

Thus the distribution of y_i conditioned on x_i and ɛ_i remains Poisson. Tornado injuries but not fatalities exhibit overdispersion, and so we report Poisson models of fatalities and negative binomial models of injuries.

We estimate the following regression models of tornado fatalities and injuries:

View Expanded

Our regression analysis includes a number of variables affecting tornado casualties in addition to measures of false alarms. Prior empirical work (Simmons and Sutter 2005, 2008) informs expectations for the variables. The models include year dummy variables to control for factors that vary across the entire nation yet are not captured by our other variables, factors like the emergence of the Internet or cell phones. In addition, tornado casualties might exhibit year-to-year variation if, for instance, a year with few tornadoes lulls residents into a false sense of security the next year. The year variable coefficients are not reported to conserve space in the tables. For a list of all variable names and brief descriptions see Table 5.

We include several different tornado warning variables. Tornado warnings for the period of our data were issued by county whereas our data points are state tornado segments from the Storm Prediction Center tornado archive. Consequently, a tornado that strikes more than one county in a state may have several valid warnings. We apply the warning for the first county in the storm path. Our main warning variables use the lead time for a warning at the start of the tornado; note that for long-path tornadoes the warning at the beginning of the storm path is less representative of the time that residents along the storm path might have to respond. We break the warning lead time down into categories, which allows the marginal effect of lead time to vary. Dummy variables lead 1–5, lead 6–10, lead 11–15, lead 16–20, lead 21–30, and lead 31+ control for the intervals, and respectively equal 1 if the lead time for the warning falls in the corresponding interval and 0 otherwise. For robustness we also use a categorical variable—warning—that equals 1 if a warning was issued for the tornado and 0 otherwise.

We include several tornado characteristic variables, all constructed from the Storm Prediction Center archive. The most important storm characteristic is the tornado’s rating on the Fujita scale of tornado damage. The ratings take on integer values from 0 to 5, with 5 being the strongest.³ We use categorical (or dummy) variables labeled F0, F1, F2, F3, F4, and F5 to represent the F scale. The variable F3, for example, equals 1 if a tornado was rated 3 on the F scale and 0 otherwise. The category dummy variables do not impose any functional relationship between F scale and casualties. More damaging tornadoes should produce more fatalities and injuries, but the categorical variables might indicate that F1 tornadoes are only slightly more deadly than F0 tornadoes, but that F4 tornadoes are much more deadly than F3 tornadoes. The regression model omits the F0 category, and so the coefficients on the included category variables indicate the effect on casualties relative to an F0 tornado.

Other variables include “length”—the tornado track length in tenths of a mile. Longer track tornadoes have the potential to kill and injure more people, so we expect a positive sign on length. We control for time of day with the dummy variables of day, evening, and night. “Day” equals 1 if the tornado occurs between 0600 and 1759, “evening” equals 1 if the tornado occurs between 1800 and 2359, and “night” equals 1 for tornadoes between 0000 and 0559 (all times local). The regressions omit night, and so the coefficients on day and evening show the effect of tornadoes at these times relative to the late-night hours. Research demonstrates that tornadoes at night are more likely to catch residents unprepared and produce more casualties, so we expect negative signs for day and evening (Ashley 2007; Ashley et al. 2008; Simmons and Sutter 2005, 2008). “Weekend” is a dummy variable equal to 1 if the tornado occurred on a Saturday or Sunday. Previous work has shown that casualties are higher on weekends. Season controls for the month of the year of the tornado and equals 1 for tornadoes during March, April, May, or June and 0 for tornadoes in any other month. Casualties are higher for tornadoes during the off season when controlling for F scale and other factors (Simmons and Sutter 2008).

We include three path characteristic variables. The path characteristic variables were based on U.S. census values for the county struck by the tornado. The values for a county for a given year are estimated by linear interpolation from the decennial censuses. Although tornadoes are small relative to counties, county-based path variables have exhibited strong explanatory power in prior regression analysis (Simmons and Sutter 2005, 2008). For tornadoes that struck more than one county, the storm value averages the county values for the year in question. The first path variable is “density”, or persons per square mile for the tornado path.⁴ Casualties should increase as population density increases. We also combine density and length because a long-track tornado through a highly populated area may affect casualties differently than an increase in each variable separately. “Mobile” is mobile homes as a percentage of county housing units.⁵ The susceptibility of mobile homes to damage from tornadic winds has long been recognized (Golden and Adams 2000; Schmidlin et al. 2001; Brooks and Doswell 2002), and confirmed in previous research. “Income” is median family income in thousands of 1999 dollars.⁶ Economists typically find that people spend more to protect themselves against risk as they become wealthier (Viscusi et al. 2000, chapter 19), and cross-national analyses of natural hazard fatalities have confirmed this negative relationship between income and hazard lethality (Anbarci et al. 2005; Kahn 2005). For tornadoes, higher-income individuals might own an in-home tornado shelter and buy National Oceanic and Atmospheric Administration (NOAA) weather radios or other emergency alert systems. Wealthier communities could be more likely to have tornado sirens and more effective emergency management and rescue services. Note, however, that tornadoes appear to be an exception to this pattern; Simmons and Sutter (2005, 2008) have found a strong, positive relationship between income and both fatalities and injuries.

5. Regression results

Regression analysis of tornado fatalities reveals that each of the six definitions of a FAR described above is a positive and statistically significant (at better than the 0.01 level) determinant of fatalities (Table 6). Figure 3 graphs the estimated impact of a one-standard-deviation (1 std dev) increase in the different definitions of the FAR on increases in fatalities, calculated using the point estimates from Table 6 and the standard deviations of the FAR variables from Table 1. Figure 3 also displays the upper and lower bounds of the 95% confidence interval (CI) for the 1-std-dev increase in FAR. An increase in the FAR increases expected fatalities by between 12% and 29% across the different definitions. The FAR variables based on two years of tornado warnings have larger effects on fatalities. The variable, “no warnings,” which controlled for cases in which no warnings had been issued in the previous 1 or 2 yr and the FAR was undefined and was included in all specifications except 2-yr state, is significant and negative for the 1-yr state and both TV definitions, with point estimates slightly larger than that of the FAR variable. Recall that tornadoes with no warnings were also coded with FAR = 1. Thus the coefficients on no warnings and the FAR must be combined to infer the impact of tornadoes occurring with no recent, local warnings. Consequently, for the 1-yr state and TV market definitions of FAR, tornadoes with no recent warnings affect fatalities similar to a FAR of 0. On the other hand, no warnings is insignificant with a small point estimate using the 1-yr WFO definition and is positive and significant for 2-yr WFO. With these definitions of FAR, tornadoes with no warnings affect casualties similarly to (or more than) a FAR of 1.⁷

The signs, significance, and magnitude of the control variables are similar to previous results (Simmons and Sutter 2005, 2008). All attain significance at the 0.10 level except some of the lead-time-interval variables. The magnitudes are quite similar across different specifications of the FAR variable. Figure 4 graphs the point estimates of warnings in the various lead-time intervals on fatalities for the 1-yr and 2-yr state FAR variables. In the figure, the effect of a tornado with no lead time is normalized to 100, so the figure displays the impact of warning lead times relative to no lead time. Warnings with lead times up to 11–15 min reduce expected fatalities when compared with no warning, and a 6–10-min lead time reduces fatalities by 46%. Yet the point estimates indicate that lead times greater than 16 min increase expected fatalities relative to a tornado with no warning lead time. The impact is statistically significant and quite large, with a warning with a 16–20-min lead time increasing fatalities by 60%–70% relative to an unwarned tornado. Although anecdotes of increasing risky activities for long–lead time tornadoes exist, such behaviors would have to be quite prevalent to more than offset the effect of some people sheltering. This surprising result is due to a handful of tornadoes with long lead times and large fatality totals.⁸ The storm-path variables density, income, and mobiles all increase expected fatalities, consistent with previous results. Timing also matters significantly for tornado fatalities. Figure 5 displays the effect of a tornado occurring during the evening or night periods relative to a day tornado. Expected fatalities from day tornadoes are normalized to equal 100, and the figure displays the increase or decrease for evening and night tornadoes based on the point estimates in Table 6 for each of the FAR definition specifications. Expected fatalities were 67% and 63% lower for tornadoes that occur during the day or evening relative to overnight tornadoes with the 1-yr FAR variables; the effects are just slightly higher for the 2-yr FAR variables. The time-of-day effects provide indirect evidence on the effectiveness of the tornado warning process, since plausibly residents will be less likely to receive and respond to a tornado warning at night. Tornadoes are also much more deadly on weekends, with expected fatalities 61%–85% higher relative to weekdays. Tornadoes occurring between March and June are less deadly relative to other months, with a reduction in fatalities of 14%–26% depending on the FAR specification. Expected fatalities increase dramatically with F scale, with an F5 tornado being 30 000 times more lethal than an F0 tornado when controlling for other determinants of fatalities. Length and the length × density interaction term also increase fatalities.

The regression models reveal that false alarms also significantly increase injuries (Table 7). Each of the six definitions of the FAR is significant at better than the 0.01 level. Figure 6 displays the effect of a 1-std-dev increase in each of the FAR variables on injuries based on the point estimates, along with the upper and lower bounds of the 95% CI. A higher FAR increases expected injuries by between 13% and 32%, very similar to the range for fatalities, with larger impacts for the WFO FARs than for the state FARs. The variable no warnings, controlling for tornadoes that occur with an undefined local, recent FAR, fails to attain significance for injuries, in contrast to the results for fatalities and rendering inferences about how residents treat rare tornado warnings as problematic. Tornado injuries occur more often than fatalities, and thus fatality analysis contains more “noise,” especially given the relatively small number of tornadoes occurring with no recent, local warnings. So in assessing the results for fatalities and injuries, the mixed results for fatalities and the consistently insignificant results for injuries suggest that residents probably treat tornadoes with no previous, recent warnings as similar to high FAR warnings.

As with fatalities, the signs, significance, and magnitude of the effect of the control variables on injuries is similar to previous research (Simmons and Sutter 2005, 2008) and is similar across the definitions of the FAR variable. The control variables affect both fatalities and injuries similarly, with the few exceptions noted below. The main differences between the injury and fatality models are for the warning-lead-time variables. Figure 7 depicts the effect of a warning lead time in each interval relative to a no-lead-time tornado (set at 100) for the 1-yr and 2-yr state specifications. Warnings reduced expected injuries for all lead-time intervals, while lead times under 15 min reduced fatalities. All of the lead-time variables are significant for all FAR specifications except in the 16–20-min interval, which attains significance for 1-yr state. The largest reduction for injuries in most cases occurs for lead 31+, which reduces injuries by about 42%, and this reduction is consistent across the FAR definitions. Reductions in expected injuries of around 40% also occur in the 11–15-min interval, and so the marginal value of a 31-min lead time relative to an 11–15-min lead time is almost zero. The most notable remaining difference between the injuries and fatalities analysis concerns weekend tornadoes, which increase injuries by less than 10% and fail to attain significance for each FAR definition, whereas tornadoes on weekends significantly increased fatalities. Figure 8 displays the impact of evening and night tornadoes on injuries, relative to a tornado during the day. The magnitude of the effect of several control variables is smaller for injuries than for fatalities, notably time of day (injuries are about 34% and 38% lower during the day or evening relative to night). The effect of F scale on injuries is substantial, with expected injuries 1300 times as great for an F5 tornado than for an F0 tornado, but is less than the effect of F scale on fatalities.

Table 8 presents fatality and injury models for the 1-yr state FAR variables for two alternative measures of warnings—a dummy variable controlling for whether a warning was in effect for a tornado and a second dummy variable “Doppler.” Doppler controls for whether the NWS WFO, with warning responsibility for the first county in the tornado path, had weather surveillance Radar 1988-Doppler (WSR-88D) installation on or before the day of the tornado and has the value of 1 if Doppler radar had been installed and 0 otherwise. Only the coefficients for the warning and false-alarm variables are reported; the control variables from the full models were all included and had the same signs and magnitude impacts as in Tables 6 and 7. The FAR variable is significant (at better than the 0.01 level) in each case with a similar magnitude effect on casualties as in Tables 6 and 7. A warning in effect decreases fatalities by 2.7% (but is not significant) and decreases injuries by a statistically significant 30%. Doppler radar reduces fatalities by 34% and injuries by 35% with both effects statistically significant. The estimated impact of Doppler radar on casualties is smaller than the 45% reduction in fatalities and 40% reduction in injuries reported in Simmons and Sutter (2005), in part because the specifications in Table 8 include the FAR, which was reduced following Doppler radar installation. Thus, part of the impact of Doppler radar here is captured in the FAR variables. The FAR variables retain their significance and magnitude, so the false-alarm effect is robust to alternative definitions of the warning variables.

6. Extensions

Our results provide strong evidence of a false-alarm effect, meaning that a higher false-alarm ratio increases casualties. This does not establish a false-alarm “problem,” because to us the term problem suggests a higher-than-optimal FAR. The tornado warning process must trade off a higher FAR for a higher POD, and that both elements of the trade-off are lethal does not imply a problem. We extend the analysis here by examining the reduction in casualties attributable to a lowered FAR over the sample period and the casualties trade-off between the POD and FAR.

7. Conclusions

This paper has investigated the impact of false alarms on tornado casualties and, by implication, response to warnings. Intuition and theory suggest that if residents place little confidence in warnings, if they think that warnings are “always false alarms,” they might ignore warnings, resulting in higher casualties. But evidence of a false-alarm effect in the field for hazard warnings generally has been elusive. We have found strong evidence that a higher local, recent FAR significantly increases tornado fatalities and injuries, justifying its use as a performance metric for the NWS. For robustness we constructed FAR variables using three different geographies for local and 1- and 2-yr windows for recent; the finding is robust for all of these plausible definitions of a local, recent FAR. The positive relationship between FAR and casualties is apparent even in tabulations of fatalities and injuries per tornado. Each different FAR variable attains statistical significance in a regression analysis, and the magnitudes are consistent across definitions, with a 1-std-dev increase in the FAR increasing expected fatalities by 12%–29% and expected injuries by 13%–32%.

Our approach has several limitations that suggest avenues for future research. First, as mentioned, we have examined casualties and not warning response directly. This allows us to use a dataset of over 20 000 tornadoes to exploit differences in warning performance across the nation to identify a false-alarm effect. But we have no direct evidence that false alarms affect the perceived credibility of warnings. Future survey or qualitative research might focus on the link between local recent false alarms and the perception of warning quality or warning response. Second, our demographic control variables are based on the counties struck by tornadoes, which are large when compared with the damage paths of individual tornadoes. While the county-based variables attain significance, and in the expected direction (with the exception of income), future research could attempt to refine the damage paths of the tornadoes. Last, as in all econometric work, omitted variables may explain some portion of the effect on casualties we attribute here to false alarms or other variables.

We stress once more in concluding that evidence that false alarms affect casualties does not imply the existence of a false-alarm “problem.” The term problem suggests that the FAR is higher than optimal given the trade-off between the FAR and POD. Given the trade-off between FAR and POD described in Brooks (2004), a reduction in FAR, given the offsetting decrease in POD, would not appear to reduce fatalities and injuries. The point estimates indicate that the false-alarm effect is almost equal to the warning effect at the observed trade-off. We find that a substantial false-alarm effect does exist, and based on this we cannot recommend maximizing the POD while ignoring the impact on false alarms.