Firm Behavior in the Face of Severe Weather: Economic Analysis between Probabilistic and Deterministic Warnings

Seth P. Howard Austin College, Sherman, Texas
University of North Carolina at Charlotte, Charlotte, North Carolina

Search for other papers by Seth P. Howard in
Current site
Google Scholar
PubMed
Close
,
Kim E. Klockow-McClain Cooperative Institute for Mesoscale Meteorological Studies/National Severe Storms Laboratory, Norman, Oklahoma

Search for other papers by Kim E. Klockow-McClain in
Current site
Google Scholar
PubMed
Close
,
Alison P. Boehmer University of California, San Diego, San Diego, California
Austin College, Sherman, Texas

Search for other papers by Alison P. Boehmer in
Current site
Google Scholar
PubMed
Close
, and
Kevin M. Simmons Department of Economics, Austin College, Sherman, Texas
National Center for Risk and Resilience, University of Oklahoma, Norman, Oklahoma

Search for other papers by Kevin M. Simmons in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

Tornadoes cause billions of dollars in damage and over 100 fatalities on average annually. Yet, an indirect cost to these storms is found in lost sales and/or lost productivity from responding to over 2000 warnings per year. This project responds to the Weather Research and Forecasting Innovation Act of 2017, H.R. 353, which calls for the use of social and behavioral science to study and improve storm warning systems. Our goal is to provide an analysis of cost avoidance that could accrue from a change to the warning paradigm, particularly to include probabilistic hazard information at storm scales. A survey of nearly 500 firms was conducted in and near the Dallas–Fort Worth metropolitan area asking questions about experience with tornadoes, sources of information for severe weather, expected cost of responding to tornado warnings, and how the firm would respond to either deterministic or probabilistic warnings. We find a dramatic change from deterministic warnings compared to the proposed probabilistic and that a probabilistic information system produces annual cost avoidance in a range of $2.3–$7.6 billion (U.S. dollars) compared to the current deterministic warning paradigm.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Kevin M. Simmons, ksimmons@austincollege.edu

Abstract

Tornadoes cause billions of dollars in damage and over 100 fatalities on average annually. Yet, an indirect cost to these storms is found in lost sales and/or lost productivity from responding to over 2000 warnings per year. This project responds to the Weather Research and Forecasting Innovation Act of 2017, H.R. 353, which calls for the use of social and behavioral science to study and improve storm warning systems. Our goal is to provide an analysis of cost avoidance that could accrue from a change to the warning paradigm, particularly to include probabilistic hazard information at storm scales. A survey of nearly 500 firms was conducted in and near the Dallas–Fort Worth metropolitan area asking questions about experience with tornadoes, sources of information for severe weather, expected cost of responding to tornado warnings, and how the firm would respond to either deterministic or probabilistic warnings. We find a dramatic change from deterministic warnings compared to the proposed probabilistic and that a probabilistic information system produces annual cost avoidance in a range of $2.3–$7.6 billion (U.S. dollars) compared to the current deterministic warning paradigm.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Kevin M. Simmons, ksimmons@austincollege.edu

1. Introduction

Tornadoes are among nature’s most powerful storms, sometimes causing notable destruction to property and lives when they occur. Over the last 10 years, Munich RE reports that in North America tornadoes and thunderstorms average $26 billion (U.S. dollars) in annual normalized total losses, adjusted for changes in prices, wealth and population (Munich RE 2020). Over the same period, casualties have averaged 113 fatalities and 1265 injuries per year (SPC Archive 2020). It is little wonder that when a tornado warning is issued, people may choose to interrupt their activities to take shelter and protect property. Responding to the warning of a potentially deadly storm is the right thing to do, but decision-makers may also note that warning response carries an opportunity cost as it requires people and businesses to forgo leisure and productive activities. When the false alarm rate of 70% is considered, that cost is amplified (Brooks and Correia 2018).

The National Weather Service (NWS) has altered protocols to address the cost. In 2007, the geography of warnings was changed from county based to storm based warnings. This change alone reduced the implicit cost of warnings by 56% (Simmons and Sutter 2011). Another potential change holds the promise of reducing cost further by providing tornado warnings alongside probabilistic information regarding the likelihood that a storm may produce a tornado, rather than a simple deterministic forecast (Rothfusz et al. 2018). This paradigm shift could carry economic consequences in a new form; rather than altering warning practices explicitly, which could carry both benefits and costs (Brooks and Correia 2018) and continue to force forecasters to make one warning decision that suits the personal safety needs of all their users, probabilistic hazard information would offer additional context to warnings and potentially allow users to optimize decisions based on their own risk tolerance. In this way, the societal burden of warning decisions (e.g., the cost of responding to repeated false alarms) could be reduced as individuals have more information at their disposal. Additionally, probabilistic hazard information is envisioned to extend well beyond the typical warning timeline, which could expand the horizon of response options.

This paper is part of the Tornado Warning Improvement and Extension Program that was authorized by the Weather Research and Forecasting Innovation Act of 2017, H.R. 353. Our goal is to examine how firms currently respond to tornado warnings and how response could change if given warnings that utilize probabilities. Using data collected from a survey administered to nearly 500 firms in the Dallas–Fort Worth Metropolitan Area, we compare the cost and response rates between the current deterministic and proposed probabilistic systems to determine if cost avoidance can be expected.

We begin with a literature review in section 2 that discusses a cost–loss model which forms the basis of our analysis. Our survey providing data for the analysis is discussed in section 3. Section 4 outlines our methodology while section 5 provides our results. Section 6 conducts a sensitivity analysis to show how results change when inputs are allowed to vary, and section 7 concludes with a short discussion on the implications of our findings.

2. Literature review

Our approach is an adaptation of a classic decision analysis problem. These types of decisions have been utilized in weather applications since the 1950s. Thompson (1952) provided the basis for a simple decision framework for forecasters in a 2 × 2 matrix where forecasts of inclement weather are compared to observed weather; this is commonly referred to as a contingency table in forecast evaluation (Doswell et al. 1990). The cells within this matrix represent frequencies between the forecasts and observations for “Weather” or “No Weather” occurrence of a critical weather event. Table 1 provides a simple illustration:

Table 1.

Cost–loss matrix.

Table 1.

In the same paper Thompson also provides a simple decision rule for taking protective action, commonly referred to as the decision threshold. If the probability P of observing a critical event exceeds the cost C of protective actions divided by potential loss L, protective measures should be taken as shown in Eq. (1):
P>C/L.
This approach was broadened in the 1970s by generalizing the model to include situations where protection not only completely eliminates the loss as suggested by Thompson but rather could be used when the loss could be reduced or eliminated (Murphy 1976). Further this generalized model could be used more widely. An example of the refined matrix is shown in Table 2.
Table 2.

Cost–loss matrix.

Table 2.

Our approach is to estimate the potential economic benefits of a probabilistic information paradigm. We adapt Table 2 from Murphy and set the decision-maker to be a firm that receives the warning.

In Table 3, C is the cost a firm suffers responding to a severe weather warning. Firms incur the response cost every time they actively respond regardless of whether a tornado occurs. The loss L is preventable loss firms incur for not responding to a warning when a severe weather event occurs. The bottom right cell is 0, which comes from a firm not responding and an event not occurring.

Table 3.

Cost–loss matrix.

Table 3.

Preventable losses L are assets that could be protected when the firm actively responds to the threat with sufficient time to protect some assets. In a survey, described in section 3, we elicit response cost from firms, in addition to responses they would take at different probability levels. To estimate preventable loss, we use the relationship between the probability at which the firm took actions of different kinds and the firm’s response cost. Equation (2) is our adaptation of Eq. (1) and shows the relationship between probability, cost, and preventable loss (Thompson and Brier 1955; Richardson 2000; Murphy et al. 1985; Katz 1993):
probability of severe weather event=response cost/preventable loss.
Equation (2) rearranged, finds preventable loss, as shown in Eq. (3). To suspend operations has a cost in terms of lost sales and/or lost productivity. Understandably, firms are reluctant to suspend operations unless the events are quite likely, has high potential preventable loss, and/or suspending operations is not very costly. For some firms with high response costs and high preventable losses, the probability of the tornado occurring may be a critical decision factor:
preventable loss=response cost/probability of the severe weather event.
This formulation for preventable loss in combination with the firm’s revealed response cost will form the basis of our methodology outlined in section 4.

3. Data

Data for the study come from a survey of firms conducted during spring/summer 2019 (Howard et al. 2021, manuscript submitted to Wea. Climate Soc.). The purpose was to examine firms’ likely responses to traditional tornado warnings and probabilistic information. Survey questions asked firms to describe their industry, experience with severe weather, their approach to responding, and the cost of responding to severe weather warnings. Distribution of the survey was through Chambers of Commerce in and near the Dallas–Fort Worth (DFW) Metropolitan area. We chose this area, for our ability to network directly with the Chambers of Commerce and facilitate survey distribution, eliciting a strong response from the local business community. Notably, firms are densely populated within the area, and North Texas has significant experience with damaging tornadoes. For example, in October 2019, a series of tornadoes with ratings up to EF3 affected the highly populated area of North Dallas destroying homes but also a church and two schools. Also, the region is growing quickly. According to Forbes, in 2018, the DFW Metro area was the third-fastest-growing region in the country.1

To generate valid measures of firm behavior for later use in the survey, a focus group was convened. The focus group was made up of nine local business owners, emergency coordinators, and security staff—all related to the process of emergency management for their respective firms. In December 2018 they gathered at the county courthouse with the help of the Grayson County Office of Emergency Management. Focus group members began with a similar survey to the one ultimately used. After completion of their survey, we conducted a group discussion on survey questions and how best to present the survey to our intended list of participants.

The most important contribution of the focus group was to calibrate a behavior ranking scale shown in Fig. 1. This scale provides a range of activities firms may take in advance of severe weather. As far as we are aware, this scale is the first attempt to document a range of firm behaviors taken during a tornado warning. Our focus group ranked each behavior allowing us to categorize a firm’s overall response as passive or active. The range for the scale was from 1 to 10 with the order of “most passive” (1) to “most active” (10) actions. As an example, a score of 7, Advising personnel, patients, clients, customers to stay away from windows, represents a firm taking more active action than a score of 3, Wait for the siren to go off. Finally, the focus group chose actions above 5 as the cutoff for active responses to the warning that would incur cost to the firm.

Fig. 1.
Fig. 1.

Behavioral ranking scale.

Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0107.1

Using a scale provides a standardized method to measure and compare behaviors under different scenarios.

The survey instrument began with background questions about the specific firm, its size, and staff training for severe weather. Along with these questions, the survey asked what sources firms use to receive severe weather information. Finally, the survey posed several scenarios to businesses and documented all actions they would take in each scenario; those scenarios included a current deterministic warning, and probabilistic forecasts of tornado occurrence at 25%, 50%, 75%, and 100% levels. Ultimately, the responses to probabilistic information are compared to the current deterministic warning.

To estimate economic consequences of a change from a deterministic to a probabilistic system we need to know the cost firms incur to respond and the response rate. Response cost comes from lost sales and/or lost productivity by responding to the potential threat. Our survey asked firms to estimate, in ranges, this cost. Survey respondents could choose from four ranges, less than $10,000, between $10,000 and $50,000, between $50,000 and $150,000, and more than $150,000.

Since not all firms respond the same way, we also need to estimate a response rate that can be applied generally. For each scenario, firms were asked to choose up to five actions they would take; the actions were presented in random order. The first scenario presented was a deterministic warning where no probabilities are provided. This type of warning is binary, a firm is either in the warned area or it is not. Deterministic warnings are the current protocol, and survey respondents were provided the warning in Fig. 2.

Fig. 2.
Fig. 2.

Deterministic warning.

Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0107.1

Probabilistic information was given next. Rather than binary, this information provides more detail about just how likely the location of the firm was to be affected by a tornado. Warning scenarios were presented in a mixed order so as to avoid ordering effects: 50%, 100%, 75%, and finally 25%. The probabilistic section of the survey contained the same questions as the deterministic section. However, the warning information presented was different. Figure 3 shows a simple prototype for the probabilistic warning information, which gives a polygon along with a dot representing the location of the firm and a brief description above.

Fig. 3.
Fig. 3.

Probabilistic warning.

Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0107.1

We define a firm as actively responding to the warning if the average of their chosen responses exceeds 5 on the behavior scale. Activities from 6 to 10 are those that require a firm to take some action in response to the warning. By using an average above 5, the firm has chosen several of the more active, and thus expensive actions.

4. Methods

Recall that a tornado warning creates a response cost to the firm since operations stop temporarily. This cost is in the form of lost sales and/or lost productivity. Deciding to respond is based on the safety of customers and employees but also on protecting assets from potential damage. Expected preventable loss is damage that could be prevented by a responding firm. Equation (4) shows the expected warning cost for each warning scenario:
expected warning cost=expected cost(^)+expected preventable loss(#),^expected response cost=weighted average response cost×response rate %,#expected preventable loss=potential loss×nonresponse frequency %×prob of occurrence×strike rate.

Our method brings together several elements to estimate the two pieces necessary to arrive at expected warning cost. Figure 4 provides a flowchart illustrating our process. Next, we calculate each piece of the equation, starting with expected response cost, using data from survey responses.

Fig. 4.
Fig. 4.

Flowchart.

Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0107.1

a. Expected response cost

As Eq. (5) indicates, expected response cost is calculated from two elements: weighted average response cost, how much it costs an average U.S. firm to respond, and response rate, the frequency which firms report taking significant/cost-bearing response actions:
expected response cost=weighted average response cost×response rate.

1) Weighted average response cost

The survey asked firms to estimate the cost to alter operations in response to a severe weather warning. As stated in section 3, answer choices for these firms were in ranges of values, less than $10,000, between $10,000 and $50,000, between $50,000 and $150,000, and more than $150,000. We use the midpoint value of the first three answer choices ($5,000, $30,000, and $100,000). The last answer choice was $150,000+, so for that range, we use the lower bound of $150,000.

Another piece of data was the size of the surveyed firms. Firms responding to the survey were larger than the national average. To estimate an accurate representation of firms’ costs, we stratified the sample ex-post. Data gathered from U.S. Census for the top 21 tornado states showed 75% of firms have 0–10 employees (small), 20% have 10–100 employees (medium), and 5% have over 100 employees (large) (U.S. Census Bureau 2016). Firms from our survey are placed in these strata based on their size. We calculated the average cost to respond for each stratum, then multiplied by the strata weight to arrive at a weighted national average response cost. Table 4 shows the results.

Table 4.

Weighted average expected response cost.

Table 4.

2) Response rate

The response rate is the number of firms that took active action in each scenario. For each scenario, firms were asked to choose up to five actions they would take in response to the information presented. Those answers are placed on the behavior ranking scale outlined in section 3. Taking the mean of those responses we arrive at an average response for each firm. If the average was greater than 5, that firm is counted as actively responding to the information. Recall that 5 was a cutoff point indicated by the focus group as beginning to incur response cost. Thus, if the average response lies beyond 5, we are assured firms are taking actions that are cost-bearing on the whole. Next, we use the number of firms that take active action and divide it by the total number of firms to calculate the response rate for each scenario. Table 5 shows the calculated response rate for each probability level and for the deterministic warning scenario.

Table 5.

Expected response rate, cost, and potential loss.

Table 5.

3) Calculation

Estimated response cost for each scenario is shown in Table 5. As an example, at 25% probability, 19% of firms take active action, and therefore 19% is multiplied by the average response cost ($12,930.44) to arrive at the expected response cost per firm of $2,678.45.

Calculations are repeated for each of the remaining scenarios. Table 5 also shows the expected response cost for each probabilistic and deterministic scenario. It is clear that as probability increases, expected response cost increases since more firms actively respond.

b. Expected preventable loss

Expected preventable loss2 is calculated from four elements. We start with the potential loss at each warning, then use the percent of nonresponding firms, the probability of tornado occurrence and finally the strike rate:
expected preventable loss=potential loss×nonresponse rate×probability of occurrence×strike rate.

1) Potential loss

Potential loss for each warning scenario is preventable damage to structures, equipment, or property that may be realized if they took passive action for that warning and a tornado occurred. In the literature review (section 2) we provided our approach to estimating potential loss through Eqs. (2) and (3). Table 5 shows estimates for potential loss calculated at each probability of a tornado occurring. Firms that respond at lower probabilities likely have more to lose so potential loss is higher.

2) Nonresponse rate

The percent of firms that do not respond is 1 − response rate.

3) Probability of occurrence

Probability of occurrence is the probability of the tornado occurring in each scenario and 1 − NWS false alarm rate for the deterministic warning scenario.

4) Strike rate

Finally, we calculate the strike rate. The intuition behind using a tornado strike rate in addition to the probability of occurrence is simply that the warned area is large compared to a typical tornado footprint. The average area of a deterministic warning is 275 square miles (H. E. Brooks 2020, personal communication). For probabilistic warnings, the probability refers to any point within 7.5 km in any direction from the warned location (Gesell 2020). Using 7.5 km as a radius gives a warned area of 68 square miles. To estimate the typical tornado footprint, we use data from the Storm Prediction Center tornado archive for years 2008–18 (SPC Archive 2020). Using the length and width of state tornado segments from the archive, the average footprint (length × width)3 is 0.34 square miles. To account for a residual debris field that may cause damage, we triple that calculation to an even 1 square mile. Providing a strike rate for a deterministic warning is (1/275) or 0.36%. For a probabilistic plume, the tornado strike rate is (1/68) or 1.47%.

5) Calculation

Before we can estimate potential loss for each scenario, we need to know what level of risk (probability of tornado occurrence) prompts a nonresponding firm to act. Recall from Table 5 that at the 25% probability level, 81% of firms fail to take active action. Of the nonresponding firms, 40% would take active action at 50%, 15% at 75%, and the remaining 45% at 100% or not at all. This breakout of when nonresponding firms would take active action allows an estimate of their potential loss. As Table 6 illustrates, for each threshold probability we multiply potential loss (from Table 5) by the proportion of firms who would act at that probability threshold. The sum of this weighted loss from each threshold is total potential loss. This is then multiplied by the percent of nonresponding firms, probability of tornado occurrence, and the strike rate to arrive at expected preventable loss.

Table 6.

The 25% probabilistic warning expected preventable loss calculation.

Table 6.

Once the strike rate is taken into consideration, expected preventable loss declines considerably. Even when a tornado occurs in the warned area, the likelihood of a firm being struck is small.

The appendix shows calculations for each of the remaining probability levels. Notably, preventable loss trends downward at higher levels of probability since there are fewer nonresponding firms. (Table A4 is a breakdown of where nonresponding firms at each probability level eventually do take active action.)

The deterministic warning expected preventable loss is slightly different from the probabilistic scenarios. In the deterministic warning, 20% of firms failed to take active action. We used firm response for the probabilistic scenarios to see at what probabilities these firms would take active action to estimate their valuation of potential loss. We found that 11% of nonresponding firms took active action at the 25% probability, 11% at 50%, 22% at 75%, and the remainder at 100% or not at all. Potential loss from these nonresponding firms is multiplied by the chance of a tornado under a deterministic warning, the percent of nonresponding firms, and the strike rate to estimate Expected Preventable Loss for all businesses in the scenario. Likelihood of a tornado occurring under a deterministic warning is 1 − false alarm rate. The national false alarm rate over the last five years is 70%. Therefore, we have a 30% chance the event occurs, and 20% of firms are not responding. Calculation for the deterministic expected loss is shown below in Table 7.

Table 7.

Deterministic warning expected loss.

Table 7.

c. Expected warning cost

The overall cost of the warning is the expected response cost of firms who respond to the warning plus expected preventable loss for nonresponding firms. Table 8 provides expected warning cost for all five warning scenarios. Expected response cost details for all warnings are in Table 5. Expected preventable loss details for the 50%, 75%, and 100% probabilistic warnings are in the appendix.

Table 8.

Expected warning cost.

Table 8.

5. Results

Estimated cost avoidance is calculated as the difference between the expected total cost for responding to a deterministic system versus a probabilistic system. In the previous section, we illustrated how we calculate components to find expected response cost and expected loss in each scenario. In this section, we bring those components together to provide our estimate of the economic impact of switching to a probabilistic warning regime.

Figure 5 provides a generic illustration of our process. Expected warning cost is the sum of expected response cost and expected preventable loss. For a deterministic warning, the number is constant; this reflects the reality that warnings can occur with a range of underlying hazard probabilities, but those probabilities are unknown to the decision-maker. However, for each probabilistic scenario, expected response cost will vary, increasing as the probability of occurrence and response frequency increases. The area to the left of the lines crossing in Fig. 5 shows cost avoidance; in this region, firms that would prefer to withhold response actions at lower probability values do so. The area to the right of where the lines cross shows a switch in the cost. This small region emerges due to a higher response rate for circumstances where tornadoes are nearly certain to exist. The last firms to respond are those with the lowest preventable loss. For them, the cost of interrupting operations exceeds the preventable loss from being hit by a tornado. Once we have estimated cost avoidance for each level of probability, the final question we must address is how often each probability level is expected to occur in reality. We do not have this information, so we assume a uniform distribution where each scenario occurs ¼ of the time. This produces an expected tornado probability that is higher than NWS performance suggests (62% rather than 30%). Given that much of the savings in our calculations were realized when firms did not take protective actions for low probability situations, by keeping expected probabilities on the high side, we are employing an assumption that generates a more conservative estimate of economic impact.

Fig. 5.
Fig. 5.

(Before calculation) expected warning cost.

Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0107.1

It should be noted that our analysis ignores the economic value of decreased casualties. An increase in the response rate should reduce casualties, so the area we treat as increased cost may show savings if the value of avoided casualties were considered. This paper, however, only considers the direct economic cost of a firm’s decision. Figure 6 illustrates the difference between deterministic and probabilistic systems’ expected response cost from our data.

Fig. 6.
Fig. 6.

(After calculation) expected warning cost.

Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0107.1

Table 9 shows expected warning cost, estimated cost avoidance, and estimated cost avoidance per scenario.

Table 9.

Per firm/scenario estimated cost avoidance.

Table 9.

Annualized cost avoidance

With estimated cost avoidance per firm/scenario, we can estimate annual national cost avoidance. This calculation shows the nationwide cost avoidance that a change to a probabilistic system may bring annually. For this, we need the number of warnings per year and the number of affected firms per year. The average number of warnings per year is provided by the NWS (H. E. Brooks 2020, personal communication). The number of affected firms must be estimated. We chose to use the average area of a warning in square miles and the number of firms per square mile in the top 21 tornado states4 (U.S. Census Bureau 2016; SPC Archive 2020). Table 10 shows our estimate of the number of affected firms per year.

Table 10.

Warning information.

Table 10.

Estimated cost avoidance per firm (Table 9) and average affected firms per year (Table 10) are multiplied together to estimate annual nationwide cost avoidance a change to a probabilistic system may bring. Using the midpoint of survey cost ranges, we find that nationally, $5.3 billion could be saved each year.

6. Sensitivity analysis

Results rely on responses from the survey regarding two variables, the cost to respond to a tornado warning and what actions the firm would take in the event of a tornado warning. There is sufficient variability in answers for both the cost and response to warrant a sensitivity analysis that examines our final results when those responses are allowed to change.

a. Response cost

Our estimates depend on firms’ cost to respond to a warning. Possible answers to the cost question in our survey were in ranges. Therefore, along with the midpoint method used throughout the paper; a low and high point method for average expected response cost is presented, providing a range of estimated cost avoidance. To provide a lower bound warning cost we took the lowest possible value for each range including zero for the first range. Our high point weighted average expected warning cost took the maximum value from each range. However, since the last range had no upper bound we again used the lower bound of $150,000. Results are shown in Table 11.

Table 11.

Annual estimated cost avoidance.

Table 11.

Results provide a range of estimated cost avoidance a probabilistic warning system could provide. Even using the low point response cost, we see an annual estimated cost avoidance of $2.35 billion, still a significant savings.

b. Response frequency

To determine a firm’s response to the warning, we asked each to list up to five actions taken when under a tornado warning. These responses were placed on the behavior ranking scale, then averaged to provide an overall response score. If the response score exceeded 5, they were considered an active respondent to the warning. Here we stress the definition of an active respondent by pushing the threshold higher. So now, to be considered an actively responding firm, we require an average response score exceeding 7 instead of 5. This would limit responding firms to those that included some of the most severe actions taken when presented with a tornado warning. Table 12 is the same as Table 11 but with 7 as the threshold for active response. We also include the sensitivity test for response cost.

Table 12.

Annual estimated cost avoidance (active > 7).

Table 12.

As Table 13 shows response drops for all scenarios but drops the most for the lowest probabilities. Yet, even stressing the definition of an active response we continue to see significant cost avoidance in the change from a deterministic to a probabilistic tornado warning system.

Table 13.

Response rate comparison.

Table 13.

Next we calculate the response rate using the maximum response instead of the average as a threshold. The final column of Table 13 shows the response rate that would be estimated if we used only firms who chose 10 as one of their response choices to that question. Using the average of responses but changing the threshold from 5 to 7 has an effect on response rates for all warnings that is almost proportionate. But when we use the maximum response, the decline in the deterministic response rate is substantially less than that of its nearest peer, the 100% warning. This increases the difference between the warning cost of a deterministic warning and all probabilistic scenarios. Consequently, using these response rates nationwide, annual cost avoidance shows an increase compared to using an average response rate of 5 as the threshold. Per firm/warning cost avoidance are now in excess of $4,000 and overall nationwide cost avoidance increases from the midpoint estimate of $5.3 to over $7 billion. We share this result, not to claim a higher cost avoidance estimate, but that our original approach of using an average score exceeding 5 as the threshold may be a reasonable estimate.

7. Conclusions

This paper uses results from a survey administered to DFW metro firms to estimate annual cost avoidance that a change to a probabilistic hazard information system could bring nationally. Results show changing the system provides positive annual estimated cost avoidance when compared to the total response cost incurred by businesses responding to the current deterministic warning system alone.

Results suggest estimated cost avoidance could have a range of $2.35–$7.6 billion annually. Even using our most conservative method, there is still an estimated cost avoidance of almost $1.44 billion. Implementing the probabilistic system should significantly reduce the cost severe weather warnings place on the economy. These results support the implementation of a policy to use a probabilistic system during severe weather events.

Our results are promising in that they provide evidence that cost avoidance should accrue from the use of probabilistic information. But the study does suffer from some limitations. First, we acknowledge that our survey is limited to the DFW area. An appropriate follow up study would be to survey firms in other parts of the country or attempt a full nationwide study. In addition, DFW is within “tornado alley,” where tornadoes are more common and we assume firms are more “weather aware” than other parts of the country. This might influence how firms responded to parts of the survey. A second limitation is that response cost in the survey is set to a range of numbers rather than a more specific number. To address this we conducted low, mid, and high point response cost methods to estimate a range of nationwide annual cost avoidance. A possible improvement would be a more detailed study on the actual realized costs firms bear responding to severe weather events. Another limitation is our stratification of responding firms. We stratified by firm size. However, another approach would be to stratify by firm type or ideally by both size and type. This could be an important determinate of the potential loss portion of our estimate. For instance, a small firm may have high value inventory they need to protect or could lose compared to a larger firm with lower value inventory. While we did provide firm type as one of the survey questions, only a few of the categories provided sufficient responses to allow for a robust determination of the cost to respond (see the appendix, Table A5). Further, our “other” category was the most chosen response. A future study that could elicit sufficient responding firms to capture this variability would provide a more accurate estimate of the potential loss firms can expect. Finally, our response rate for probabilistic information was from a visualization of the warning based solely on the spatial attributes of the firm’s location to the approaching storm. A more sophisticated study using laboratory simulations where both space and time could be altered may provide further insight into the role space and time play in a firm’s decision to respond.

Acknowledgments

Funding was provided by NOAA/Office of Oceanic and Atmospheric Research under NOAA–University of Oklahoma Cooperative Agreement NA16OAR4320115, U.S. Department of Commerce.

Data availability statement

Data from the survey as well as the survey itself is kept on servers at Austin College and is available with permission by the authors.

APPENDIX

Tables for All Scenarios

The appendix is provided to illustrate tables for all scenarios in addition to the examples provided in the text (see Tables A1A5).

Table A1.

50% expected preventable loss.

Table A1.
Table A2.

75% expected preventable loss.

Table A2.
Table A3.

100% expected preventable loss.

Table A3.
Table A4.

Firms proportion breakdown.

Table A4.
Table A5.

Distribution of firm type.

Table A5.

REFERENCES

  • Brooks, H. E., and J. Correia Jr., 2018: Long-term performance metrics for National Weather Service tornado warnings. Wea. Forecasting, 33, 15011511, https://doi.org/10.1175/WAF-D-18-0120.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Doswell, C. A., III, R. Davies-Jones, and D. L. Keller, 1990: On summary measures of skill in rare event forecasting based on contingency tables. Wea. Forecasting, 5, 576585, https://doi.org/10.1175/1520-0434(1990)005<0576:OSMOSI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gesell, I. W., 2020: Verification of the tornado and lightning plumes and evaluation of a new kernel for the tornado and lightning plumes. M.S. thesis, University of Oklahoma, 123 pp.

  • Katz, R. W., 1993: Dynamic cost–loss ratio decision-making model with an autocorrelated climate variable. J. Climate, 6, 151160, https://doi.org/10.1175/1520-0442(1993)006<0151:DCLRDM>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Munich RE, 2020: Natural disaster losses are trending upward. Accessed 18 February 2021, https://www.munichre.com/en/risks/natural-disasters-losses-are-trending-upwards/thunderstorms-hail-and-tornados.html.

  • Murphy, A. H., 1976: Decision-making models in the cost–loss ratio situation and measures of the value of probability forecasts. Mon. Wea. Rev., 104, 10581065, https://doi.org/10.1175/1520-0493(1976)104<1058:DMMITC>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., R. W. Katz, R. L. Winkler, and W. Hsu, 1985: Repetitive decision making and the value of forecasts in the cost–loss ratio situation: A dynamic model. Mon. Wea. Rev., 113, 801813, https://doi.org/10.1175/1520-0493(1985)113<0801:RDMATV>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Richardson, D. S., 2000: Skill and relative economic value of the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 126, 649667, https://doi.org/10.1002/qj.49712656313.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rothfusz, L. P., R. Schneider, D. Novak, K. Klockow-McClain, A. E. Gerard, C. Karstens, and T. M. Smith, 2018: FACETs: A proposed next-generation paradigm for high-impact weather forecasting. Bull. Amer. Meteor. Soc., 99, 20252043, https://doi.org/10.1175/BAMS-D-16-0100.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Simmons, K. M., and D. Sutter, 2011: The Economic and Societal Impact of Tornadoes. Amer. Meteor. Soc., 282 pp.

    • Crossref
    • Export Citation
  • SPC Archive, 2020: Storm Prediction Center WCM page. Accessed 18 February 2021, http://www.spc.noaa.gov/wcm/.

  • Thompson, J. C., 1952: On the operational deficiencies in categorical weather forecasts. Bull. Amer. Meteor. Soc., 33, 223226, https://doi.org/10.1175/1520-0477-33.6.223.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Thompson, J. C., and G. W. Brier, 1955: The economic utility of weather forecasts. Mon. Wea. Rev., 83, 249253, https://doi.org/10.1175/1520-0493(1955)083<0249:TEUOWF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • U.S. Census Bureau, 2016: U. S. Census 2016 County Business Patterns. Accessed 18 February 2021, https://www.census.gov/programs-surveys/cbp.html.

2

Expected preventable loss also depends on things like the intensity of the tornado and the type of firm.

3

Length is given in miles, but width is given in yards so width was adjusted to miles.

4

To find the average number of firms per square mile, we used the U.S. Census Business Activity report of 2016. This provided the number of firms for each state. To determine the top tornado prone states we used data from the Storm Prediction Center archive. Finally, the square mileage of each state is again provided by the U.S. Census.

Save
  • Brooks, H. E., and J. Correia Jr., 2018: Long-term performance metrics for National Weather Service tornado warnings. Wea. Forecasting, 33, 15011511, https://doi.org/10.1175/WAF-D-18-0120.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Doswell, C. A., III, R. Davies-Jones, and D. L. Keller, 1990: On summary measures of skill in rare event forecasting based on contingency tables. Wea. Forecasting, 5, 576585, https://doi.org/10.1175/1520-0434(1990)005<0576:OSMOSI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gesell, I. W., 2020: Verification of the tornado and lightning plumes and evaluation of a new kernel for the tornado and lightning plumes. M.S. thesis, University of Oklahoma, 123 pp.

  • Katz, R. W., 1993: Dynamic cost–loss ratio decision-making model with an autocorrelated climate variable. J. Climate, 6, 151160, https://doi.org/10.1175/1520-0442(1993)006<0151:DCLRDM>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Munich RE, 2020: Natural disaster losses are trending upward. Accessed 18 February 2021, https://www.munichre.com/en/risks/natural-disasters-losses-are-trending-upwards/thunderstorms-hail-and-tornados.html.

  • Murphy, A. H., 1976: Decision-making models in the cost–loss ratio situation and measures of the value of probability forecasts. Mon. Wea. Rev., 104, 10581065, https://doi.org/10.1175/1520-0493(1976)104<1058:DMMITC>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., R. W. Katz, R. L. Winkler, and W. Hsu, 1985: Repetitive decision making and the value of forecasts in the cost–loss ratio situation: A dynamic model. Mon. Wea. Rev., 113, 801813, https://doi.org/10.1175/1520-0493(1985)113<0801:RDMATV>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Richardson, D. S., 2000: Skill and relative economic value of the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 126, 649667, https://doi.org/10.1002/qj.49712656313.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rothfusz, L. P., R. Schneider, D. Novak, K. Klockow-McClain, A. E. Gerard, C. Karstens, and T. M. Smith, 2018: FACETs: A proposed next-generation paradigm for high-impact weather forecasting. Bull. Amer. Meteor. Soc., 99, 20252043, https://doi.org/10.1175/BAMS-D-16-0100.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Simmons, K. M., and D. Sutter, 2011: The Economic and Societal Impact of Tornadoes. Amer. Meteor. Soc., 282 pp.

    • Crossref
    • Export Citation
  • SPC Archive, 2020: Storm Prediction Center WCM page. Accessed 18 February 2021, http://www.spc.noaa.gov/wcm/.

  • Thompson, J. C., 1952: On the operational deficiencies in categorical weather forecasts. Bull. Amer. Meteor. Soc., 33, 223226, https://doi.org/10.1175/1520-0477-33.6.223.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Thompson, J. C., and G. W. Brier, 1955: The economic utility of weather forecasts. Mon. Wea. Rev., 83, 249253, https://doi.org/10.1175/1520-0493(1955)083<0249:TEUOWF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • U.S. Census Bureau, 2016: U. S. Census 2016 County Business Patterns. Accessed 18 February 2021, https://www.census.gov/programs-surveys/cbp.html.

All Time Past Year Past 30 Days
Abstract Views 465 0 0
Full Text Views 510 173 12
PDF Downloads 431 119 14