1. Introduction and motivation
Emergency management primarily consists of the four main components of the comprehensive emergency model introduced in the mid-1970s (Gruntfest and Weber 1998; Waugh 1994). The model divides emergency management functions into mitigation, preparedness, response, and recovery (Weaver et al. 2014; Waugh 1994; McLoughlin 1985; Petak 1985). These four elements include a significant part of the existing philosophical vocabulary in emergency management and are widely recognized as the starting point for policy and program design administration (Waugh 1994; Petak 1985). Emergency management remains an essential component in most economies, as both natural and human-made disasters cannot be prevented.
Public safety officials with emergency management responsibilities have the vital task of alerting the public of impending inclement weather by being liaisons between local National Weather Service (NWS) forecasters and the public. Public safety official is a broad term that for the purposes of this study refers to the variety of decision-makers that make weather-related decisions to protect specific groups of people. These decision-makers include, but are not limited to, emergency managers, law enforcement officials, fire officials, school officials, health officials, and city/county/state/tribal government officials. While public safety officials are the focus of this paper, the term emergency manager (EM) will be used predominantly in this first section because much of the previous research on weather decision-makers was focused specifically on EMs.
EMs make time-sensitive decisions to warn the public of threats and link the NWS and other stakeholders (Baumgart et al. 2008). Additionally, EMs direct and protect mobile storm spotters responsible for providing “ground truth” data to better assess weather hazards (Baumgart and Bass 2006; Baumgart et al. 2006). Thus, the role of EMs cannot be overemphasized, particularly in parts of the country prone to hazardous weather events, such as the southern Great Plains.
Not only does the NWS work with EMs to ensure their target audience receives their message but they also rely on EMs to ensure their audience understands, believes, and responds appropriately to their message (Weaver et al. 2014). Weaver et al. (2014) also argued that this connection is critical because local EMs are responsible for identifying risks and managing vulnerabilities within a community. EMs must carry out these tasks in a way that promotes coordination, flexibility, and professionalism. Thus, one can infer that many economies risk losing lives and property if this link is broken.
As early as 1985, researchers have studied the impact of emergency response on life and property saved (Petak 1985). However, these decision-makers have been a challenging group to study because of a vast diversity in job roles, responsibilities, jurisdiction, training, tools available, responsibilities, experience level, and decision-making methods (Baumgart et al. 2008; Baumgart and Bass 2006; Baumgart et al. 2006; Doswell et al. 1999; Lusk et al. 1990).
Perhaps the greatest challenge for officials responsible for public safety is integrating information from multiple sources with different content, structure, and update rates (Baumgart and Bass 2006). According to Morss and Ralph (2007), EMs have to follow a general decision cycle involving the identification of an event’s increasing potential of occurrence. This is usually accomplished by deciding whether an event is likely or imminent, choosing to activate or deactivate their Emergency Operation Center (EOC), and completing after-action reports to document what happened. It is essential to recognize that EMs have reported that 78% of all disasters that had taken place were weather-related, yet, EMs are usually not meteorologists (Weaver et al. 2014). Additionally, the authors reported that only 31% of training classes EMs attended were weather-related classes. While we recognize that training on a diverse set of disasters is important (e.g., terrorism, industrial or hazardous chemical incidents, transportation, and construction projects), one might wonder if the training on integrating weather information into decision-making is adequate.
The NWS has undergone many transformations, including the most recent “modernization and associated restructuring” (MAR; Friday 1994). The MAR fundamentally changed the NWS field office structure to ensure that rapid detection, timely forecasts, and warnings are delivered to the public while providing greater interaction with local communities (Friday 1994). These transformations have helped to connect NWS information to emergency managers and ensure the integration of weather information to decision-making (Uccellini and Ten Hoeve 2019). Many NWS offices hold integrated warning team meetings to improve communication, understanding, and relationships between EMs and NWS.
Although there is a proliferation of weather data and resources freely available, Hocker et al. (2018) argue that officials should never be expected to make well-informed decisions from weather data without appropriate training. Addressing how EMs could overcome this challenge, Baumgart and Bass (2006) stressed the need to use tools that present weather information in multiple formats and with different update rates in simulated real-time to help EMs make informed decisions.
There has always been a myriad of resources for EMs to use (Morss and Ralph 2007; Morris et al. 2002), although there is often a disconnect with dissemination, response, and planning (Hoss and Fischbeck 2016; Weaver et al. 2014; Morris et al. 2002, 2001). In effect, not only is there a need to improve EMs’ training and decision support tools, but an in-depth investigation is necessary to understand the needs and cues EMs require to make assessments and decisions during severe events (Hocker et al. 2018; Baumgart et al. 2008). Additionally, Baumgart et al. (2006) developed a descriptive decision-making model of weather information usage, weather assessments, and decisions made during severe weather to help EMs plan for events. The authors believed that tools and models helped develop better decision support systems, improved training, and provided insight on how innovative weather information could affect the role of emergency managers in protecting the public.
The Oklahoma Mesonet has served as a model network for the establishment of statewide monitoring networks across the United States and abroad because of its accuracy, consistency, validation, broad scope, range of variables collected, monitoring and analysis, and maintenance of near-complete records (Ziolkowska et al. 2017; McPherson et al. 2007; Fiebrich et al. 2006; Shafer et al. 2000). As an outreach program of the Oklahoma Mesonet, Oklahoma’s First-response Information Resource System using Telecommunications (OK-First) program was introduced in the mid-1990s to fill a service void in Oklahoma’s weather warning system by bridging the gap between the NWS and public safety officials, especially in rural Oklahoma (Morris et al. 2002, 2001). While other weather-related training programs existed at that time (e.g., National Weather Service Spotter Training, Cooperative Program for Operational Meteorology, Education, and Training, etc.), none addressed a critical unmet need of decision-makers in Oklahoma—the need for real-time weather and radar data as well as training for proper interpretation of that information. The OK-First program was the response to those needs and paired training with data from the newly commissioned Oklahoma Mesonet. The program was created to provide routine weather and radar training from degreed meteorologists, access to critical real-time weather data, follow-up support, and networking opportunities. Now nearly 25 years later, OK-First has grown to more than 800 active members and continues to provide training and password-protected data access to its members.
Morris et al. (2002) reported that the OK-First program had become a catalyst for change in many local governments. Because of high-quality weather data tools, regular training classes, and continual follow-up support, local officials are increasingly empowered to make impacts such as close bridges during floods, save property in wildfires, improve evacuations after hazardous spills, and protect audiences at outdoor events (Morris et al. 2002). Participation in the OK-First program is competitive, restricted to officials with public safety responsibilities (for profit entities are not eligible), and dependent on successful completion of a 4-day OK-First Certification class. Additionally, once a member is certified in the OK-First program, they are required to attend recertification classes no less than once every 18 months. The OK-First program has shown immense value in providing nonscientific audiences with complex meteorological information when paired with well-designed, relevant, and routine training (Hocker et al. 2018; Morris et al. 2002).
The literature suggests a need to synthesize and provide weather information and training to EMs who can then apply the training to make informed decisions; however, this work has been unsustained or rarely present at the state level in other regions. Other programs, such as NC-First in North Carolina, have been successful; however, they were discontinued due to a lack of funding. While efforts to train decision-makers on weather topics have grown in recent years via Federal Emergency Management Agency certified classes offered by the National Disaster Preparedness Training Center, continual weather training is still vitally needed at the local level. Information collected from participants of the OK-First program offers a unique opportunity to analyze the impacts of an over 20-yr sustained program.
Whereas other scientists have focused on using surveys and interviews to assess this unique public safety community (e.g., Hoss and Fischbeck 2016; Weaver et al. 2014; Baumgart et al. 2008; Morris et al. 2002, 2001; James et al. 2000), this research more broadly assesses impacts through three separate instruments: surveys, Google Analytics, and NOAA’s Storm Events Database. This paper fills a gap in the literature to assess the perceptions, beneficiaries, and applications of weather support systems for decision-making for public safety officials. This research also reveals specific decisions public safety officials made and an estimate of potential savings. In an effort to assess the true impact of a weather decision support system, this study seeks to answer the following research questions:
-
What products and tools are most important to OK-First users to allow them to accomplish their roles in their various jurisdictions?
-
How routinely do public safety officials report using OK-First tools?
-
Are there geographic variations in OK-First tool use?
-
What are the key decisions public safety officials make to save lives and property, and how often are they made?
-
How can the self-reported economic impact of a decision support system be quantified?
2. Data and methods
This study used a mixed approach of instruments and methods to collect and analyze data from three key sources: 1) survey data, 2) Google Analytics data, and 3) NOAA’s Storm Events Database. The choice to use a mixed approach grew from informal meetings with participants during OK-First classes and emergency management conferences, meetings with stakeholders, and email and phone conversations with managers of different state mesonet systems across the country.
a. Survey design and population
A survey was developed through feedback and meetings with numerous public safety officials and stakeholders across Oklahoma. The participant survey was designed to allow the researchers to gather as much data as possible, without being too much of a time burden for the participants. Although the total number of questions was 24, the maximum number of questions answered with logic branches was 18. The survey consisted of closed and open-ended questions. Participants responded to questions on OK-First engagement, decisions made using OK-First, and other resources used.
1) Implementation
The survey was distributed following Institutional Review Board approval (number 11814). The anonymous survey link was shared with all 812 active OK-First members. Between 3 April 2020 and 11 May 2020, 280 respondents completed the survey, which yielded a 34% response rate.
2) Background of members
Before focusing on several of the key questions posed in the survey, it is important to briefly gain an understanding of the background of the survey respondents. Of the 280 respondents, 59% had participated in the OK-First program for 1–5 years, 23% had been in the program for 6–10 years, and 18% had been in the program for more than 10 years. For the number of OK-First classes attended by the respondents, approximately 67% had attended 1–5 OK-First classes, 22% had attended 6–10 classes, and 11% had attended more than 10 classes.
The geographic distribution of respondents covered all nine Oklahoma climate divisions (Fig. 1), with the greatest share (39%) coming from central Oklahoma. Central Oklahoma is the most populous part of the state and has the greatest concentration of public safety agencies (including local, county, and state agencies). Table 1 depicts the job type each survey participant identified. Nearly 80% of the respondents selected emergency manager, fire official, or law enforcement. A few write-in job types for the “other” category included storm spotter, public health official, engineer, jail administrator, and amateur radio operator.
Geographic distribution of survey respondents by climate division.
Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0086.1
Job types of OK-First Survey participants (N = 280; survey respondents were allowed to select more than one job type).
b. NOAA’s Storm Events Database
The Storm Events Database archive is maintained by NOAA (https://www.ncdc.noaa.gov/stormevents/ftp.jsp) and contains data from January 1950 to October 2020 (at the conclusion of the analysis). The database documents 1) events whose measured intensity exceeded criteria for severe storms (i.e., winds ≥ 58 mi h (26 m s−1), hail ≥ 1in. (2.54 cm) in diameter, or a tornado); 2) the occurrence of storms and other significant weather phenomena having enough intensity to cause loss of life, injuries, significant property damage, and disruption to commerce; 3) rare, unusual weather phenomena that generate media attention, such as snow flurries in South Florida; and 4) other significant meteorological events, such as record maximum or minimum temperatures or precipitation that occur in connection with another event (https://www.ncdc.noaa.gov/stormevents/).
Reports included in the NOAA storm database are each vetted by location, and only officially sanctioned terms are used. This ensures a comprehensive dataset for recorded and reported events across the nation. The exploration of the Storm Events Database provided key details of storms in Oklahoma and when they occurred. The storm events were then compared with the OK-First website statistics to understand the resources public safety officials used to make decisions in Oklahoma.
c. Google Analytics data
Google Analytics (https://analytics.google.com) is a platform that collects data and compiles them into a variety of reports on user interactions. Different organizations use web analytics to understand the interaction between a web page and visitors to the web page (Phippen et al. 2004). Web analytics involves collecting, monitoring, measuring, analyzing, and reporting web usage data to meet organizations’ goals and users’ expectations (Hasan et al. 2005). Web analytics data are collected by Google and can be compared over time (Kent et al. 2011). The OK-First program provides a decision support web page for situational awareness during hazardous weather events and human-made disasters. The OK-First program is currently on its fourth-generation decision support web page, hereinafter referred to as the OK-First Weather Briefing page. The page was launched in 2016. Kent et al. (2011) argued that 3–6 months is long enough for most organizations with a well-established web presence to see meaningful data in their Google Analytics statistics. The results presented here utilized 5 years of analytics (2016–20) to provide a robust period from which to understand frequency of use of the OK-First Weather Briefing page, especially during hazardous weather events.
In Google Analytics terminology, a session refers to a group of web hits recorded for a user in each period. Hits are defined as user interactions with the OK-First Weather Briefing page. A user can generate one or more sessions in Google Analytics. In general, the number of users and sessions follow the same trend. These data help to identify both when and how many users interacted with the OK-First Weather Briefing page. The Google Analytics data were compared with storm events reported and survey results of how participants use data from the OK-First program.
3. Results
a. Assessment of perceptions
Members of the OK-First program include a diverse group comprising more than 10 different job types (Table 1). Over 75% of participants considered the training, technology, follow-up support, and OK-First connections as very important for their duties within their various jurisdictions (Fig. 2). Ninety percent of participants also highlighted other specific services provided by the OK-First program as being very important to accomplish their work duties. Respondents specified these other services as refresher training, continual updates, radar knowledge, latest data, and safety of first responders at incidents (not shown).
Responses to the survey question “In terms of importance to your work duties, how do you value the services you receive from OK-First?” (Note that respondents could only select one response for each category.)
Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0086.1
The OK-First program provides a variety of tools to users, including an iPhone and Android application (hereinafter referred to as the Mesonet app), a publicly available web page (https://www.mesonet.org), the Weather Briefing page (password protected), and a stand-alone radar viewer for Windows computers called RadarFirst (password protected). Figure 3 summarizes the frequency of use for each tool. The Mesonet app, Oklahoma Mesonet web page, and OK-First Weather Briefing page are used most commonly on a daily basis. Of note, more than 50% of the respondents indicated that they use the Mesonet app every day.
Frequency of use of OK-First tools. (Note that “Only when there is a weather event” overlaps with other categories.)
Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0086.1
Regional differences existed in the most commonly used OK-First tools particularly on a daily basis and during weather events. Those results are shown in Fig. 4, reinforcing that no single tool can fulfill the weather information needs for a diverse community of public safety officials. Figure 4a shows that the top two most-used tools on a daily basis included the Mesonet app and the Mesonet web page. This result suggests that OK-First users across most areas need basic weather information each day, which can be satisfied through the app and the publicly available Oklahoma Mesonet web page. However, when there is a weather event, tool use by OK-First users changes significantly (Fig. 4b). Respondents statewide consistently reported that their most frequently used tool was either the RadarFirst software or the OK-First Weather Briefing page on a weather event day. Both of these tools are password protected and only available to OK-First users who are up-to-date on their training requirements. These two tools provide interactive radar products (i.e., the RadarFirst software) and a customizable multipanel web page that can display a variety of products that a user chooses, such as interactive radar data, Oklahoma Mesonet maps, NWS warnings, satellite data, and more (i.e., the OK-First Weather Briefing page). With a higher level of customization and interactivity, these tools better support critical decision-making during short-fused situations. Given the usage of different tools on varying time scales (e.g., daily versus during weather events), these results illustrate that public safety officials cannot rely on a single one-size-fits-all weather tool but require a portfolio of tools to remain weather aware.
Geographic variations of (a) the two most-used OK-First tools on a daily basis and (b) the most-used tool during a weather event.
Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0086.1
b. Assessment of beneficiaries (public safety officials and communities)
As discussed in the previous sections, the OK-First program provides training, technology, follow-up support, and connections to the program’s participants. The users can then use OK-First tools and information to make decisions for their various jurisdictions (Fig. 5). Of course, these decisions generally depended on the time of year and the specific role of a public safety official in their jurisdiction. Over 60% of participants recalled making decisions on floods, winter weather, and severe storms at least 1–5 times within a 12-month period. For respondents who reported making weather-related decisions 6–10 times in the previous 12 months, the most common cause was for severe thunderstorms (26%), closely followed by winter weather (21%). For those respondents who reported making decisions more than 10 times, severe weather, fire weather, and others were noted by more than 25% of respondents.
Number of times that OK-First tools and information were used to make decisions over a 12-month period. (Note that respondents could only select one frequency category for each weather event type.)
Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0086.1
When surveyed on the specific decisions that were made (Fig. 6), over 54% of respondents used their OK-First training to inform an official (e.g., mayor, superintendent, county commissioner, etc.) so that they could make community decisions based on an impending weather event. Forty-nine percent (49%) prepositioned resources for an imminent weather event. Forty-four percent (44%) initiated or provided a “significant weather potential” update/notice to public officials, the general public, and so on. Thirty-eight percent (38%) reported that they sounded an outdoor warning system for their community. Thirty-three percent (33%) helped to make a closure or delayed opening decision (school, government offices, etc.), and 32% generated an adequate response after an impactful weather event. Between 8% and 24% canceled or postponed a minor community event (e.g., parade or group meeting), included an OK-First trained staff member in the Emergency Operations Center to provide weather support during an event (scheduled or emergency), canceled or postponed a major community event (e.g., outdoor concert or sporting event), closed a road or rerouted traffic for an impending flooding event or other significant weather factors (e.g., rerouting first responders because of active tornado path of travel), assisted with a law enforcement investigation, or helped with a construction project (road, building, etc.). These decisions made could be categorized as mitigation, preparedness, response, or recovery (e.g., the four main components of the comprehensive emergency management model), but most responses fit within the categories of response and recovery (Fig. 6 and the appendix).
Decisions made using OK-First tools and information.
Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0086.1
There was also a statistically significant relationship between the number of years respondents participated in the program (p = 0.0001), the number of classes participants attended (p = 0.026), and their ability to make decisions for their jurisdiction. The survey results show that most OK-First participants use the tools regularly and make decisions frequently. While it is possible that this correlation can be attributed to the fact that those public safety officials with the time and interest to devote to their jobs take additional classes and thereby make better decisions, survey results indicate that the more classes users took, the more likely they were to make weather-related decisions for their jurisdiction. Additionally, survey results revealed that the more recently a respondent took an OK-First training class, the more likely they were to make a weather-related decision (not shown).
Approximately 32% of respondents who made decisions over the previous 12 months were able to estimate how much the OK-First program saved their jurisdiction. Fifty percent (50%) of those 32% indicated that the OK-First program saved their community over $10,000 over the past 12 months, 35% reported savings of $1,000–$10,000, and 15% saved under $1,000. These respondents also provided specific examples of events that supported their estimates (see the appendix). Participants who found it challenging to estimate costs also provided explanations including not knowing how to estimate the cost of losing a life and the difficulty in calculating “damage avoided” cost.
c. Assessment of applications
1) Google Analytics
(i) Trends in the diurnal use of tools
By analyzing Google Analytics data from 2016 to 2020 for the OK-First Weather Briefing page, it was possible to explore the behavior of OK-First members and how they accessed the tool. In general, the period of greatest website usage mirrored the traditional workday hours of 0700–1600 LT as shown in Fig. 7a, with the heaviest usage during the 0700–1000 LT time frame (accounting for 36% of their time on the page). A secondary maximum in website usage was also observed in the 1300–1500 (LT) time frame. Website usage declined in the evening, though it reached a relative maximum around 2000 LT before declining significantly during the overnight hours. These data suggest that users typically check weather data early in their workdays to plan for the day, followed by continued monitoring of information in the afternoon before tapering off late in the day into the overnight hours. Because Google Analytics resets all sessions at midnight, this likely accounts for the slight increase in sessions at the midnight hour.
OK-First Weather Briefing page sessions by (a) hour, (b) day of the week, and (c) month.
Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0086.1
(ii) Trends in the weekday and weekend use of the tools
Google Analytics data show that OK-First participants use the Weather Briefing page considerably more on weekdays than on the weekends (Fig. 7b). The number of sessions reaches a peak on Tuesday and starts a steady decline until it reaches a minimum number of sessions on Sunday. Tuesday and Wednesday had the most sessions for the period and represent 19% and 18% of user visits, respectively. Analysis of NOAA’s Storm Events Database for the same time frame also found that Tuesday had the highest number of events reported with 1295 events. Meanwhile, Sunday had the lowest number of events reported in NOAA’s Storm Events Database with 993 events. Although it is unclear why a significantly different number of storm events were recorded on Tuesdays versus Sundays, it appears that the usage of OK-First (as measured by Google Analytics) is driven by the frequency of weather events, even on a daily basis over the course of the week.
(iii) Trends in the monthly use of the tools
An analysis of sessions on the Weather Briefing page revealed that peak web usage occurred between March and May. March reported the most sessions at 16% of the overall sessions (Fig. 7c). The overlap of multiple weather hazards during March (e.g., wildfires, severe storms, and sometimes winter storms) likely contributed to it having the most sessions of any month in the year. The data are consistent with the NOAA Storm Events Database. The three months from March to May contributed 51% of all reported storms. The month of July had the lowest number of sessions representing only 5% of the total number of sessions.
(iv) Assessment of highest use days
Data from Google Analytics reveal that the highest number of sessions for the OK-First Weather Briefing page was 897 sessions on 20 May 2019, corresponding with a high risk of severe storms, tornadoes, and flooding that day. The next three highest usage days between 2016 and 2020 were 30 April 2019, 26 April 2016, and 17 April 2019, which also corresponded to severe storms associated with risks of tornadoes, hail, and flooding. Spikes in data usage reinforced that public safety officials relied heavily on the OK-First Weather Briefing page to prepare for and manage high-impact events. Days with the lowest page views typically corresponded to weekends, holidays, or months outside peak storm season.
2) NOAA’S Storm Events Database and OK-First Weather Briefing page
In an effort to provide a detailed analysis of the correlation between the OK-First Weather Briefing page usage and actual storm events archived in the NOAA Storm Events Database, Fig. 8 focuses on the March 2019 to April 2020 period. This 12-month period corresponds with the period survey respondents were asked to report their decisions made using the OK-First tools and training. Figure 8 shows a significant positive correlation between sessions recorded on the OK-First Weather Briefing page and weather events (r = 0.7; p < 0.001). This further reinforces that OK-First tool usage increases as impactful weather events increase.
Scatterplot of weather events reported in the NOAA Storm Events Database and OK-First Weather Briefing page sessions from March 2019 to April 2020 in Oklahoma.
Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0086.1
4. Discussion
The long-standing history of the OK-First program provided an ideal opportunity to reveal a deeper understanding of the public safety community and their use of weather information (Baumgart et al. 2008; Baumgart and Bass 2006; James et al. 2000; Morris and Duvall 1999). It was found that public safety officials have different job roles, responsibilities, jurisdictions, training, tools available, responsibilities, and experience levels. Similar to Hoss and Fischbeck (2016), Weaver et al. (2014), and Doswell et al. (1999), it was found that public safety officials operate under varying organizational structures and different staff makeup including volunteers, part-timers, or retirees. For example, public safety officials often perform diverse roles they may not be trained for, such as coordinating rescue efforts or performing tasks such as supporting firefighters or directing traffic. Additionally, public safety officials join programs such as OK-First to find ways to collaborate with others and gain new tools and resources to prepare for disasters. For instance, over 90% of public safety officials felt that the training and technology provided by the OK-First program were very important in helping them to perform their various duties in their jurisdictions.
This research aimed to expand the results of Baumgart et al. (2008), who also studied decision-making by emergency managers. OK-First participants reported using OK-First tools to make various time-sensitive decisions to warn the public of threats, including severe weather. The daily use of OK-First tools, even in the absence of extreme weather, shows the importance of having continual access to trusted, high-quality weather information (Hoss and Fischbeck 2016; Weaver et al. 2014; Baumgart et al. 2008, 2006; Morss and Ralph 2007; Baumgart and Bass 2006). The decisions that OK-First participants made were diverse and included informing an official, coordinating with schools concerning weather-related closures, interrupting local television programming for emergency alerts, opening community storm shelters, communicating with first respondents, and activating tornado sirens. These results demonstrate that programs like OK-First can enhance situational awareness and increase the ability of officials to make weather-related decisions. Over 85% of survey respondents reported making decisions for their jurisdictions based on OK-First training and tools. Similar to the findings of Hoss and Fischbeck (2016), OK-First participants indicated the importance of education and training, which led to confidence in making decisions.
Additionally, consistent with the findings of Stadler and Fiebrich (2018), analyzing Google Analytics data alongside weather event reports from NOAA’s Storm Events Database provided valuable insights. These results show that decision-makers use the tools most heavily on weather event days, which supports the views of previous studies that provided training, technology, products, and tools, public safety officials can utilize meteorological information to mitigate disasters and improve preparedness (Hunemuller 2010; Rayner et al. 2005; Penning-Rowsell et al. 2000; Golden and Adams 2000; Parker and Handmer 1998).
This estimate does not capture savings made by participants who responded with savings between $1,000 and $10,000 nor does it capture specific savings greater than $10,000. This paper is a first step in estimating the value of information provided by weather decision programs such as the OK-First program. To fully explore the economic value of the OK-First program, however, nonmarket valuation techniques such as contingent valuation, avoided costs (averting behavior), or evaluation of the difference in value between the outcome realized with access to the OK-First system and the outcome realized without access to the OK-First system would be necessary (and are beyond the scope of this paper).
5. Summary
Several key perceptions, beneficiaries, and applications of a weather decision support system for Oklahoma’s public safety officials were revealed through surveys, Google Analytics, and the NOAA’s Storm Events Database. Among the various findings, the following are the most significant based on the research questions. First, public safety officials considered the different products and tools provided by the OK-First program as very important to accomplish their roles in their various jurisdictions. Second, public safety officials reported using and integrating OK-First tools in their workflow on a daily basis. Third, although the Mesonet App was the most frequently used OK-First tool, there were geographic variations with OK-First tool use. Fourth, public safety officials took a diverse set of actions based on their access to the OK-First tools and data, with the most frequent being to inform decision-makers. In fact, over 25% of respondents indicated making 10 or more weather-related decisions for fire, severe weather, and winter weather during a 12-month period. Fifth, self-reported cost savings based on decisions made by OK-First participants were estimated to be over $1.2 million for a 12-month period.
The roles of public safety officials in Oklahoma have evolved significantly, particularly with respect to managing Oklahoma’s year-round threat of hazardous weather. With this evolution, there has been a shift from responding to events to preparing for and preplanning for them. Overwhelmingly, the OK-First program has demonstrated that public safety officials can make well-informed decisions to protect life and property and save precious resources when adequately trained and provided with trustworthy and timely weather information. However, these dividends require a committed and sustained approach that is consistently funded to be successful. With the proper funding and requisite meteorological and technical expertise, other hazardous-weather-prone states could reap cost-saving benefits similar to what Oklahoma has experienced over the last 25 years.
Acknowledgments.
The authors thank Dr. Jadwiga Ziolkowska for her assistance during the project’s initial stages and Drs. Aimee Franklin, Michael Wimberly, and Scott Greene for their guidance and support. The authors also thank the managers of the various statewide automated networks, the OK-First community, and two anonymous reviewers who provided significant and helpful input.
Data availability statement.
NOAA’s Storm Events Database is openly available (https://www.ncdc.noaa.gov/stormevents/ftp.jsp). The Google Analytics data are password protected. Further information about the data is available at the Oklahoma Mesonet (https://doi.org/10.15763/dbs.mesonet). This study received approval from the University of Oklahoma Institutional Review Board (IRB) to collect human subjects’ data. In line with that approval, only researchers identified in the IRB package have access to the survey data collected.
APPENDIX
Selected Examples of Decisions Made by OK-First Members
This appendix contains 12 quotations about decisions that led to alerts, warnings, postponements, cancellations, and more effective planning.
We have cancelled sporting events, graduation, changed bus routes, postponed investigation on [a] river, advised commissioners to stage equipment for road clearing, closed city parks due to flooding or lightning, Initiated warning systems for high wind events, tornados, and flooding, evacuated neighborhoods for flooding.
I have made the decision to sound outdoor warning sirens and to warn the school district to take shelter. [We] sometimes [use it] to notify the general public using social media. We use it for fire danger prediction.
Specific dates are difficult to come up with; however, we use Mesonet/OK-First every weather event to include Fire Weather. We have postponed yearly events that occur in May for sever[e] storms approaching and have assisted in school closure decisions with Board Members. We were able to prepare for historic floods [t]hat occurred in the Sperry area. Just recently we used the OK-First Radar during the storms that came through northeast Oklahoma [on] Wednesday and made decisions based off the data. We use mesonet and Radar first with every event, weather potential, and daily operations.
Just this year, sirens were sounded in our county/city (Choctaw/Hugo) because of watching the weather on OK-First and the Decision page also. Every year since I began my job as EM Director for Choctaw County, I always use the OK-First [] and leave the screens are up for everyone to see the weather in the EOC (especially severe thunderstorms, tornados). Also, I like the features that can be used in events that can back up the dates, down to the minute and print out for other departments. The NWS, Mesonet, OK-First employees are helpful and help with problems that we may be having or reading the maps.
Postponed [] a parade due to significant wind threat. Notified County Official of fire risk on a pipeline construction project with dry conditions and wind threat. Same pipeline weeks later with heavy rain in the past and more rain in weather models, [threatening] to go under [the] road near stream bed and bridge and had a potential for washout of [the] bridge approach.
We used the data to place and confirm spotter information, advise spotters and community leaders of track and timing for warning information, and preplacement of response resources to the event. Luckily they were not needed. We also use the information to plot track information for damage assessment. This particular tornado lifted just as it came into our county and produced only minor damage here.
On the morning of 9 October 2018, an EF1 tornado went through Midwest City. I had attended OK-First training about two weeks prior. I can state that I was able to recognize radar imagery (BVEL) that demonstrated a couplet—this gave me enough forewarning to send out an emergency alert through a text-message and phone based system to my organization (Rose State College). While the tornado ultimately did not reach my workplace, it did damage and flip cars at a local strip mall. The RadarFirst software was also instrumental in observing severe weather that had occurred on the evening of 26 August 2019. Through the RadarFirst software, I was able to observe the storm as it approached the College (where we have residents and housing) and issue an emergency notification.
May 2019—Had a Level 2 Dam Emergency involving a high hazard private dam. The continued risk of severe weather and heavy rainfall in the upstream watershed producing excessive runoff threatened to [raise] the lake level, thereby increasing hydrostatic pressure on the damaged area and causing water to flow into the damaged spillway and further eroding the dam. Weather data/information was used to make decisions to preposition emergency resources, determine on-scene staffing levels, and make preparations to evacuate[] downstream residents.
I use O[K]-First pretty much all the time, but specifically I used it last week on the 11th of April to monitor the severe weather as it passed across our county. I used it to give a prewarning for hail and high winds, and then I used the Mesonet tools to anticipate staging spotters.
Graduation 2019 Altus–downburst/microburst or isolated tornado event the night of graduation. Postponed for some time but was allowed to complete the event after the storm passage. The school where it was held sustained wind damage and debris damage to the buildings, cars, and patrons. Some injuries were sustained in the event. [It] [c]ould have been worse.
Two weeks ago, Cotton County received flash flooding after 2–3 days of heavy rain. It helped to have this resource to not only plan for road closures of flood-prone areas but also helped in getting the information to the appropriate people on the road crews.
[T]he flooding of the Salt Fork river at [T]onkawa . . . the radar estimates of Grant County rainfall, the QPF for the next two days. I have read the river since 1982 and never had a flood last as long as the one 20 May–22 June 2019. [T]he Salt Fork was above flood stage and had three crests. Had it not been for OK-First and all its links to get current forecast and stage information people could have made some bad decisions.
REFERENCES
Baumgart, L. A., and E. J. Bass, 2006: Presenting information in simulated real-time to support part-task weather scenarios. 2006 IEEE Int. Conf. on Systems, Man and Cybernetics, Taipei, Taiwan, Institute of Electrical and Electronics Engineers, 5209–5214, https://doi.org/10.1109/ICSMC.2006.385135.
Baumgart, L. A., E. J. Bass, B. Philips, and K. Kloesel, 2006: Emergency management decision-making during severe weather. Proc. Hum. Factors Ergon. Soc. Annu. Meet., 50, 381–385, https://doi.org/10.1177/154193120605000336.
Baumgart, L. A., E. J. Bass, B. Philips, and K. Kloesel, 2008: Emergency management decision making during severe weather. Wea. Forecasting, 23, 1268–1279, https://doi.org/10.1175/2008WAF2007092.1.
Cho, J. Y. N., and J. M. Kurdzo, 2019: Weather radar network benefit model for tornadoes. J. Appl. Meteor. Climatol., 58, 971–987, https://doi.org/10.1175/JAMC-D-18-0205.1.
Cho, J. Y. N., and J. M. Kurdzo, 2020: Weather radar network benefit model for nontornadic thunderstorm wind casualty cost reduction. Wea. Climate Soc., 12, 789–804, https://doi.org/10.1175/WCAS-D-20-0063.1.
Doswell, C. A., III, A. R. Moller, and H. E. Brooks, 1999: Storm spotting and public awareness since the first tornado forecasts of 1948. Wea. Forecasting, 14, 544–557, https://doi.org/10.1175/1520-0434(1999)014<0544:SSAPAS>2.0.CO;2.
Fiebrich, C. A., D. L. Grimsley, R. A. McPherson, K. A. Kesler, and G. R. Essenberg, 2006: The value of routine site visits in managing and maintaining quality data from the Oklahoma Mesonet. J. Atmos. Oceanic Technol., 23, 406–416, https://doi.org/10.1175/JTECH1852.1.
Friday, E. W., 1994: The modernization and associated restructuring of the National Weather Service: An overview. Bull. Amer. Meteor. Soc., 75, 43–52, https://doi.org/10.1175/1520-0477(1994)075<0043:TMAARO>2.0.CO;2.
Golden, J. H., and C. R. Adams, 2000: The tornado problem: Forecast, warning, and response. Nat. Hazards Rev., 1, 107–118, https://doi.org/10.1061/(ASCE)1527-6988(2000)1:2(107).
Gruntfest, E., and M. Weber, 1998: Internet and emergency management: Prospects for the future. Int. J. Mass Emerg. Disasters., 16, 55–72.
Hasan, L., A. Morris, and S. Probets, 2005: Using Google analytics to evaluate the usability of E-commerce sites. Human Centered Design, M. Kurosu, Ed., Lecture Notes in Computer Science, Vol. 5619, Springer, 697–706.
Hocker, J. E., A. D. Melvin, K. A. Kloesel, C. A. Fiebrich, R. W. Hill, R. D. Smith, and S. F. Piltz, 2018: The evolution and impact of a meteorological outreach program for public safety officials: An update on the Oklahoma Mesonet’s OK-First program. Bull. Amer. Meteor. Soc., 99, 2009–2024, https://doi.org/10.1175/BAMS-D-17-0100.1.
Hoss, F., and P. Fischbeck, 2016: Increasing the value of uncertain weather and river forecasts for emergency managers. Bull. Amer. Meteor. Soc., 97, 85–97, https://doi.org/10.1175/BAMS-D-13-00275.1.
Hunemuller, T. J., 2010: Review and analysis of the National Weather Service River Forecasts for the June 2008 eastern Iowa floods. M.S. thesis, Dept. of Civil and Environmental Engineering, University of Iowa, 98 pp., https://doi.org/10.17077/etd.4nmfqfrh.
James, T. E., P. O. Long, and M. A. Shafer, 2000: An independent evaluation of the OK-FIRST decision-support system. Second Symp. on Environmental Applications, Long Beach, CA, Amer. Meteor. Soc., 40–47.
Kent, M. L., B. J. Carr, R. A. Husted, and R. A. Pop, 2011: Learning web analytics: A tool for strategic communication. Public Relat. Rev., 37, 536–543, https://doi.org/10.1016/j.pubrev.2011.09.011.
Klockow, K. E., R. A. Mcpherson, and D. S. Sutter, 2010: On the economic nature of crop production decisions using the Oklahoma Mesonet. Wea. Climate Soc., 2, 224–236, https://doi.org/10.1175/2010WCAS1034.1.
Lawson, M., 2019: Using economics to support climate adaptation. Headwaters Economics, https://headwaterseconomics.org/equity/climate-change/economics-climate-adaptation/.
Lusk, C., T. Stewart, K. Hammond, and R. Potts, 1990: Judgment and decision making in dynamic tasks: The case of forecasting the microburst. Wea. Forecasting, 5, 627–639, https://doi.org/10.1175/1520-0434(1990)005<0627:JADMID>2.0.CO;2.
McLoughlin, D., 1985: A framework for integrated emergency management. Public Adm. Rev., 45, 165–172, https://doi.org/10.2307/3135011.
McPherson, R. A., and Coauthors, 2007: Statewide monitoring of the mesoscale environment: A technical update on the Oklahoma Mesonet. J. Atmos. Oceanic Technol., 24, 301–321, https://doi.org/10.1175/JTECH1976.1.
Morris, D. A., and C. Duvall, 1999: Real-time, an evaluation of the use of weather data by public-safety agencies. Preprints, 15th Int. Conf. on Interactive Information and Processing Systems for Meteorology, Oceanography, and Hydrology, Dallas, TX, Amer. Meteor. Soc., 380–383.
Morris, D. A., K. C. Crawford, K. A. Kloesel, and J. M. Wolfinbarger, 2001: OK-FIRST: A meteorological information system for public safety. Bull. Amer. Meteor. Soc., 82, 1911–1923, https://doi.org/10.1175/1520-0477(2001)082<1911:OFAMIS>2.3.CO;2.
Morris, D. A., K. C. Crawford, K. A. Kloesel, and G. Kitch, 2002: OK-FIRST: An example of successful collaboration between the meteorological and emergency response communities on 3 May 1999. Wea. Forecasting, 17, 567–576, https://doi.org/10.1175/1520-0434(2002)017<0567:OFAEOS>2.0.CO;2.
Morss, R. E., and F. M. Ralph, 2007: Use of information by National Weather Service forecasters and emergency managers during CALJET and PACJET-2001. Wea. Forecasting, 22, 539–555, https://doi.org/10.1175/WAF1001.1.
Parker, D. J., and J. W. Handmer, 1998: The role of unofficial flood warning systems. J. Contingencies Crisis Manage., 6, 45–60, https://doi.org/10.1111/1468-5973.00067.
Penning-Rowsell, E. C., S. M. Tunstall, S. M. Tapsell, and D. J. Parker, 2000: The benefits of flood warnings: Real but elusive, and politically significant. Water Environ. J., 14, 7–10, https://doi.org/10.1111/j.1747-6593.2000.tb00219.x.
Petak, W. J., 1985: Emergency management: A challenge for public administration. Public Adm. Rev., 45, 3–7, https://doi.org/10.2307/3134992.
Phippen, A., L. Sheppard, and S. Furnell, 2004: A practical evaluation of web analytics. Internet Res., 14, 284–293, https://doi.org/10.1108/10662240410555306.
Rayner, S., D. Lach, and H. Ingram, 2005: Weather forecasts are for wimps: Why water resource managers do not use climate forecasts. Climatic Change, 69, 197–227, https://doi.org/10.1007/s10584-005-3148-z.
Shafer, M. A., C. A. Fiebrich, D. S. Arndt, S. E. Fredrickson, and T. W. Hughes, 2000: Quality assurance procedures in the Oklahoma Mesonetwork. J. Atmos. Oceanic Technol., 17, 474–494, https://doi.org/10.1175/1520-0426(2000)017<0474:QAPITO>2.0.CO;2.
Stadler, S. J., and C. A. Fiebrich, 2018: Web hits on the Oklahoma Mesonet. American Association of Geographers Annual Meeting 2018, New Orleans, LA, American Association of Geographers, 059, https://aag.secure-abstracts.com/AAG%20Annual%20Meeting%202018/abstracts-gallery/12811.
Uccellini, L. W., and J. E. Ten Hoeve, 2019: Evolving the National Weather Service to build a Weather-Ready Nation: Connecting observations, forecasts, and warnings to decision-makers through impact-based decision support services. Bull. Amer. Meteor. Soc., 100, 1923–1942, https://doi.org/10.1175/BAMS-D-18-0159.1.
Waugh, W. L., 1994: Regionalizing emergency management: Counties as state and local government. Public Adm. Rev., 54, 253–258, https://doi.org/10.2307/976728.
Weaver, J., L. C. Harkabus, J. Braun, S. Miller, R. Cox, J. Griffith, and R. J. Mazur, 2014: An overview of a demographic study of United States emergency managers. Bull. Amer. Meteor. Soc., 95, 199–203, https://doi.org/10.1175/BAMS-D-12-00183.1.
Ziolkowska, J. R., 2018: Economic value of environmental and weather information for agricultural decisions—A case study for Oklahoma Mesonet. Agric. Ecosyst. Environ., 265, 503–512, https://doi.org/10.1016/j.agee.2018.07.008.
Ziolkowska, J. R., and Coauthors, 2017: Benefits and beneficiaries of the Oklahoma Mesonet: A multisectoral ripple effect analysis. Wea. Climate Soc., 9, 499–519, https://doi.org/10.1175/WCAS-D-16-0139.1.
