• Baumgart, L., , and Bass E. J. , 2006: Presenting information in simulated real-time to support part-task weather scenarios. Proc. 2006 Int. Conf. on Systems, Man, and Cybernetics, Taipei, Taiwan, IEEE, 5209–5214.

  • Baumgart, L. A., , Bass E. J. , , Philips B. , , and Kloesel K. , 2006: Emergency management decision-making during severe weather. Proc. 50th Annual Meeting of the Human Factors and Ergonomics Society, San Francisco, CA, Human Factors and Ergonomics Society, 381–385.

    • Crossref
    • Export Citation
  • Brown, S., , Archer P. , , Kruger E. , , and Mallonee S. , 2002: Tornado-related deaths and injuries in Oklahoma due to the 3 May 1999 tornadoes. Wea. Forecasting, 17 , 343353.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Doswell C. A. III, , , Moller A. R. , , and Brooks H. E. , 1999: Storm spotting and public awareness since the first tornado forecasts of 1948. Wea. Forecasting, 14 , 544557.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • FEMA, cited. 2007: Tornado activity in the United States. Federal Emergency Management Agency. [Available online at http://www.fema.gov/plan/prevent/saferoom/tsfs02_torn_activity.shtm.].

  • Hunter, D. R., , Martinussen M. , , and Wiggins M. , 2003: Understanding how pilots make weather-related decisions. Int. J. Aviat. Psychol., 13 , 7387.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lusk, C. M., , Stewart T. R. , , Hammond K. R. , , and Potts R. J. , 1990: Judgment and decision making in dynamic tasks: The case of forecasting the microburst. Wea. Forecasting, 5 , 628639.

    • Search Google Scholar
    • Export Citation
  • McLaughlin, D., and Coauthors, 2007: Engineering Research Center for Collaborative Adaptive Sensing of the Atmosphere. Annual Report Year 4, Vols. I and II, NSF Cooperative Agreement EEC-0313747, University of Massachusetts—Amherst, 747 pp.

    • Search Google Scholar
    • Export Citation
  • Morris, D. A., , Crawford K. C. , , Kloesel K. A. , , and Wolfinbarger J. M. , 2001: OK-FIRST: A meteorological information system for public safety. Bull. Amer. Meteor. Soc., 82 , 19111915.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morris, D. A., , Crawford K. C. , , Kloesel K. A. , , and Kitch G. , 2002: OK-FIRST: An example of successful collaboration between the meteorological and emergency response communities on 3 May 1999. Wea. Forecasting, 17 , 567576.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morss, R. E., , and Ralph F. M. , 2007: Use of information by National Weather Service forecasters and emergency managers during CALJET and PACJET-2001. Wea. Forecasting, 22 , 539555.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • OCS, cited. 2007: OK-FIRST. Oklahoma Climatological Survey. http://okfirst.mesonet.org/.].

  • SPC, cited. 2007: Tornadoes and deaths by year and month, 1950–1999. NOAA/Storm Prediction Center. [Available online at http://www.spc.noaa.gov/archive/tornadoes/ustdbmy.html.].

  • Stewart, T. R., , and Lusk C. M. , 1994: Seven components of judgmental forecasting skill: Implications for research and the improvement of forecasts. J. Forecasting, 13 , 579599.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stewart, T. R., , Pielke R. Jr., , and Nath R. , 2004: Understanding user decision making and the value of improved forecasts. Bull. Amer. Meteor. Soc., 85 , 223235.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wiggins, M., , and O’Hare D. , 1995: Expertise in aeronautical weather-related decision making: A cross-sectional analysis of general aviation pilots. J. Appl. Exp. Psychol., 1 , 305320.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery

    Conceptual framework of the decision-making process of EMs during a severe weather event. Each box represents a component of either the environmental system, the information system, or the perceptual and cognitive system of the EM. The solid arrows represent inputs from one system component to another and the dashed arrows represent influences between system components.

  • View in gallery

    Tool for presenting weather information in simulated real time. The top two panels contain (left) base velocity and (right) base reflectivity data. NWS text information is on the bottom left panel and the legend is in the bottom right.

  • View in gallery

    Number of EMs who used or requested different information sources during the simulated weather scenario.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 309 309 22
PDF Downloads 237 237 13

Emergency Management Decision Making during Severe Weather

View More View Less
  • 1 University of Virginia, Charlottesville, Virginia
  • | 2 University of Massachusetts—Amherst, Amherst, Massachusetts
  • | 3 University of Oklahoma, Norman, Oklahoma
© Get Permissions
Full access

Abstract

Emergency managers make time-sensitive decisions in order to protect the public from threats including severe weather. Simulation and questionnaires were used to capture the decision-making process of emergency managers during severe weather events. These data were combined with insights from emergency manager instructors, National Weather Service (NWS) forecasters, and experienced emergency managers to develop a descriptive decision-making model of weather information usage, weather assessments, and decisions made during severe weather. This decision-making model can be used to develop better decision support tools, improve training, and to understand how innovative weather information could potentially affect emergency managers’ role of protecting the public.

Corresponding author address: Ellen J. Bass, Dept. of Systems and Information Engineering, University of Virginia, 151 Engineer's Way, Charlottesville, VA 22904. Email: ejb4n@virginia.edu

Abstract

Emergency managers make time-sensitive decisions in order to protect the public from threats including severe weather. Simulation and questionnaires were used to capture the decision-making process of emergency managers during severe weather events. These data were combined with insights from emergency manager instructors, National Weather Service (NWS) forecasters, and experienced emergency managers to develop a descriptive decision-making model of weather information usage, weather assessments, and decisions made during severe weather. This decision-making model can be used to develop better decision support tools, improve training, and to understand how innovative weather information could potentially affect emergency managers’ role of protecting the public.

Corresponding author address: Ellen J. Bass, Dept. of Systems and Information Engineering, University of Virginia, 151 Engineer's Way, Charlottesville, VA 22904. Email: ejb4n@virginia.edu

Keywords: Severe storms

1. Introduction

a. Emergency management during severe weather

The United States experiences some of the most severe thunderstorms and violent tornadoes anywhere in the world (FEMA 2007). Between 1950 and 1999, tornadoes accounted for an average of 89 fatalities each year (SPC 2007). On 3 May 1999 alone, 58 tornadoes occurred in Oklahoma, leaving 45 people dead and 600 injured (Brown et al. 2002). One critical role that emergency managers (EMs) play is to alert the public of approaching severe thunderstorms, in order to save lives, especially in the tornado-prone areas of the central United States. While the public obtains a great deal of weather information from the media, EMs provide an essential link between local National Weather Service (NWS) weather forecasters, who issue severe weather warnings, and the public. Emergency managers call schools and hospitals, interrupt local television programming, open storm shelters, communicate with first responders, and in some cases, sound local storm sirens (Doswell et al. 1999). Many EMs are also responsible for directing and protecting mobile storm spotters who provide visual “ground truth” data to help with understanding the weather situation.

With the recent catastrophic events such as Hurricane Katrina in 2005, emergency management and public safety have received increased focus by the public and academia. However, EM information use and decision making during severe weather has not been well documented. Morss and Ralph (2007) suggest that EMs follow a general decision cycle as hazardous weather events unfold. This cycle depicts a series of major decision points involving the identification of increased potential for an event, the decision that an event is likely or imminent, the decision to activate or deactivate their emergency operations center, and completion of postevent wrap-up, which could take months to years following an event (Morss and Ralph 2007). Throughout a severe weather event, EMs use a variety of information sources, particularly radar products and NWS forecasts (Morss and Ralph 2007; Morris et al. 2002).

To improve emergency manager training and decision support tools, more in-depth investigation is needed to understand the specific sources of information that EMs use, the perceptual cues they seek from these sources to make weather assessments, and the corresponding decisions they make during severe weather events. Emergency managers are very difficult to study due to lack of standardization in the jurisdictional nature of their work. Emergency managers have access to different weather products, computational assets, and Internet bandwidth (Bass et al. 2008, manuscript submitted to J. Emerg. Manage.). They have varied sources of training, tool sets, and methods for making decisions. Further, specific responsibilities and experiences of EMs vary by jurisdiction (ranging from full-time state employees to part-time unpaid volunteers).

In Oklahoma, there has been a focus on understanding and improving emergency management during weather-impacted situations. The Oklahoma Climatological Survey (OCS) has developed an Internet-based decision support system using an all-hazards approach called OK-FIRST (Morris et al. 2001). Table 1 presents a representative set of products available to EMs who participate in OK-FIRST. Weather information contained in the decision support system is based on the Oklahoma Mesonet (automated stations collecting surface observations such as temperature, humidity, and wind speed), 15 Doppler radars located in Oklahoma and surrounding states, and weather forecast products from the NWS and computer models. Low-resolution geographical information is also provided including county borders, highways, and major rivers (OCS 2007; Morris et al. 2001). In addition to the decision support system developed by the OCS, EMs may also gather information during severe weather from other sources, such as the media, other Internet sites, and storm spotters.

OCS has trained over 400 emergency managers in nearly 200 jurisdictions. OK-FIRST’s initial training consists of 4–5 days of instruction aimed at building computer literacy and knowledge of meteorological fundamentals, including working through case studies with data from actual weather events. This initial training is followed by a minimum of 1-day-long refresher courses every 18 months. The refresher courses often cover updates to the software in addition to providing new materials on data interpretation. Follow-up support between workshops is also available to OK-FIRST participants (Morris et al. 2001).

b. Development of an initial EM descriptive decision-making model

Although the OK-FIRST decision support system and training are model programs for enhancing emergency manager performance during weather events, there is no formal detailed decision-making model to drive their designs. Through interviews with OK-FIRST instructors, NWS forecasters, and the more senior EMs in the OK-FIRST program, an initial descriptive decision-making model of Oklahoma EM information use and decision making was developed (Table 2). The EM information use and decisions are divided into four temporal phases of severe weather leading up to an actual event (such as a tornado). These four phases, based on actions taken by the Storm Prediction Center (SPC) and the local NWS weather forecast office, are (I) prestorm, (II) severe weather watch, (III) severe weather warning, and (IV) severe weather event. Table 2 describes these phases and depicts some of the information sources used in the third column and decisions typically made in the fourth column during the associated weather phase. The initial model reveals that radar data (reflectivity and velocity), spotter reports, the media, and updates from the local NWS office are typically used by EMs in multiple weather phases. Typical decisions illustrated in the initial model include determining where and when to deploy spotters during the severe weather watch phase and determining if and when to sound storm sirens during the severe weather warning phase.

A conceptual framework adapted from Lusk et al. (1990) and Stewart et al. (2004) was developed to aid in the visualization of the EM decision-making process during a severe weather event (Fig. 1). The decision-making cycle can be mapped through three main systems: the environmental, information, and cognitive systems, all of which are involved in weather-related decision making (Stewart and Lusk 1994). The environmental system consists of the actual weather event in addition to the time of day and the location of the event. The information system collects and presents information about the event, and the cognitive system consists of decision makers gathering perceptual cues from the prior two systems in order to make an assessment and associated decision. The solid arrows in Fig. 1 represent inputs from one system component to another and the dashed arrows represent an influence that one system component has on another. The framework illustrates that there are many factors beyond the input of perceptual cues gathered from information sources that influence EM weather assessments and decisions, including the time of day, location of the storm, training, experience, approach to risk, and knowing what assessments and decisions the NWS has made. Additionally, the framework also illustrates that two-way communication between local NWS forecasters and EMs occurs through the steps of gathering perceptual cues, assessing the weather, and making decisions.

The objective of this study was to validate the model and framework through direct interactions with EMs. Additionally, our goals were to understand the perceptual cues that EMs extract from information sources to assess weather situations and to understand what conditions or assessments lead to their specific decisions during severe thunderstorms and tornadic activity. The method involved a combination of a simulated weather scenario, surveys, and feedback from OK-FIRST instructors. The results of this study will help to inform the development of better EM decision support tools and training as well as to uncover how innovative weather information could potentially affect EMs’ role of protecting the public.

2. Method

a. Participants

Collaborative Adaptive Sensing of the Atmosphere (CASA) is a research project introducing new weather radar technologies (McLaughlin et al. 2007). A 45 km × 45 km test bed of four radars has been installed in southwest Oklahoma and has been in operation since March 2006. At the time of this study, 20 EMs worked in the testbed region. Eight of these EMs volunteered to participate in the study. Three OK-FIRST-trained EMs to the north of the region also volunteered. These 11 participants had emergency management experience ranging from 1 to 33 yr (median: 8 yr). For the purpose of this analysis, 7 of the 11 were experienced (defined as having ≥3 yr of experience) and 4 were considered as novice (defined as having <3 yr of experience).

b. Apparatus

1) Severe weather decision-making questionnaire

A severe weather decision-making questionnaire, consisting of 50 questions, was divided into five sections. Four sections involved the weather phases described in the initial descriptive decision-making model (Table 2) and one was for demographics. The weather phase sections probed weather product usage during a typical severe thunderstorm event for 13 information sources available to OK-FIRST program participants (first column in Table 1). Subjective measures of value (Likert scale: 1–5), frequency of use, and information obtained were requested for each weather product in each phase. The phase sections also included questions regarding typical phase-specific decisions and actions taken during a typical severe thunderstorm. Participants were asked what specific conditions are necessary to initiate certain decisions, such as sounding the sirens (severe weather warning and severe weather event phases) and spotter deployment (severe weather watch and severe weather warning phases). Questions regarding information integration and geographic details were asked in the severe weather warning and severe weather event sections only. Additional question details are available online (http://www.sys.virginia.edu/ejb/WAF/DMQuestionnaire.pdf).

2) Tool for presenting simulated weather scenarios

Currently, few weather simulators exist to create weather products (such as radar data) and access to them is limited. For these reasons, archived weather information is often presented in simulated scenarios, or case studies, to study weather-related decision making. A custom tool was developed to present time-coordinated radar images, NWS text products, and Geographical Information System (GIS) data from archived weather events in simulated real time (see Baumgart and Bass 2006 for more details). Severe weather data for this study consisted of NWS text products and archived radar images (base reflectivity and base velocity from the lowest radar elevation angle) from an event on 29 May 2001 in Tulia, Texas, an area unfamiliar to the EMs.

The tool’s human–computer interface provided display panes for radar reflectivity and radar velocity images and a display pane for text (see Fig. 2). Each pane included current and minutes-old weather information. During a scenario, participants could resize the images by either using the mouse directly on the image or by using the associated zooming buttons below each image. Previous images were available using a scroll bar below the image or by using the animation button. In either case, previous and current images were displayed in time sequence.

The event chosen for the scenario included a supercell thunderstorm, a notorious and consistent severe weather producer. The Oklahoma EMs in this study are familiar with supercells and their simultaneous threats of straight-line winds, large hail, heavy rainfall and tornadoes. Using a supercell thunderstorm supported the study of a wide variety of potential products used and decisions made by EMs. Because of the multiple threats presented by the supercell, it also allowed for an investigation of how EMs prioritized these threats.

Ultimately, this supercell thunderstorm did not produce a severe weather event in the limited domain used in the simulation. However, the storm did produce 1.25-in. hail upstream of the area and 2.50-in. hail downstream of the jurisdiction used in the simulation. This storm did not produce a tornado, although other supercells in the vicinity did produce tornadoes. The participants were provided the information that the local NWS weather forecast office had issued a severe thunderstorm warning and that large hail and damaging winds were expected.

3) Post-scenario questionnaire

The post-scenario questionnaire collected additional information regarding the weather assessments made and associated decisions that were (or may have been) made during the scenario. Participants responded with free-form text to address the use of different radar products, the value of GIS data, opinions on radar data display implications, and what decisions were reached during the scenario. Additional question details are available online (http://www.sys.virginia.edu/ejb/WAF/Post1Questionnaire.pdf).

c. Procedure

The 11 volunteers participated in one of three scheduled sessions. For observation, at least one analyst was paired with each participant. After signing a consent form and receiving an introduction, participants completed the simulated weather scenario and the post-scenario questionnaire. Participants were instructed to think aloud throughout the scenario and the scenarios were audio-recorded. Contact with local NWS forecasters and storm spotters was not simulated, but participants were instructed that thinking aloud included telling the analyst when and why they would contact the NWS and spotters and what the objective of the contact was. The tool presented the weather scenario, which consisted of the severe weather warning and severe weather event phases only. These are phases where the public needs to take protective action in a timely manner. To increase workload, the severe weather decision-making questionnaire served as a secondary task during the scenario. If not completed, participants were asked to finish it before leaving. The participants then received further training and completed a second scenario not discussed herein (see Baumgart et al. 2006 for more detail).

d. Data collection and analysis

In addition to collecting responses from both the severe weather decision-making questionnaire and the post-scenario questionnaire, audio recordings from the simulated weather scenario were transcribed and coded. Dependent variables derived from the transcriptions included information sources used, cross validations between information sources, weather assessments reported, decisions made, and human factors–related comments. These variables would provide additional insight into how EMs interpret information from multiple sources, assess the weather, and make appropriate decisions that affect the public during severe weather. Information provided by the participants in the post-scenario questionnaire was associated with actual information used and decisions made during the scenario. This was used to validate the findings from the scenario transcripts and to provide stronger support for the refinements to the initial descriptive model for the severe weather warning and severe weather event phases only.

To refine the initial model, phase-specific criteria were used for validating and adding an information source. The following criteria for used the prestorm and severe weather watch phases:

  • The median perceived value of the information source was 4 or 5 (valuable or very valuable) in the severe weather decision-making questionnaire.
  • Five or more EMs reported using the information source at least once per day in the severe weather decision-making questionnaire.

The following criterion was used for the severe weather warning and severe weather event phases:

  • The information source was used to gather information by at least two EMs during the scenario.

A summary of the perceptual cues sought in the information sources accessed and the weather assessments or conditions associated with specific decisions made during each severe weather phase was also added to the model. This information was gathered from both the severe weather decision-making questionnaire and the actions or comments made during the scenario.

3. Results

Results regarding the validation and refinement of the initial descriptive decision-making model of emergency management in Oklahoma during severe weather are presented in two sections: 1) weather information use and perceptual cues gathered and 2) weather assessments and decisions made. Potential EM weather perception and assessment issues that were revealed during the simulated weather scenario are also presented. This is followed by a detailed analysis of one experienced and one novice EM. Statistical results are reported as significant for α = 0.05 and as a trend for α = 0.1.

a. Weather information use and perceptual cues gathered

Figure 3 shows the total number of EMs who used or requested specific information sources during the simulated weather scenario. All EMs used radar reflectivity and the one EM who did not use radar velocity was an EM with less than 1 yr of experience. In the severe weather decision-making questionnaire, EMs rated radar data (both reflectivity and velocity) and also spotter reports as “very valuable,” independent of severe weather phase. Additional transcript data from the simulated weather scenario indicated that the radar velocity source was used to gather perceptual cues regarding the location and intensity of wind shear, and the storm-relative velocity (SRV) source was seen as more useful for locating cyclonic rotation inside the storm. For gathering cues regarding the size and presence of hail, the vertically integrated liquid (VIL) product was their primary source of information.

Rows 1 and 2 in Table 3 summarize the number of information sources used and the number of cross validations between information sources for both experienced and novice EMs. Many of the information sources were used to confirm or validate pre-existing perceptual cues gleaned from prior information sources used as seen during the simulated weather scenario. For example, spotter reports and TV reports were used as “ground truth” or to confirm what EMs had perceived to be rotation from the radar data. Communication with local NWS forecasters was also used to confirm and assist in gathering perceptual cues from radar information sources and to make associated weather assessments.

Aside from the fact that EMs used or requested products not probed by the severe weather decision-making questionnaire, there was little difference between the questionnaire responses and the scenario commentary with respect to perceptual cues sought from the information sources. However, in the questionnaire data, EMs did not report using radar reflectivity to gather perceptual cues about the “big picture” (e.g., looking for approaching storms to the west or south), whereas six EMs mentioned using this source for such cues during the scenario.

Table 4 reflects revisions of the initial model for information sources typically used, their perceived values, and the perceptual cues EMs seek when accessing the source during the severe weather warning phase only. The data were combined from both the scenario transcripts and the severe weather decision-making questionnaire. The information sources that do not have an associated perceived value are sources that at least two EMs stated they would use during the scenario, but were not included in the questionnaire.

b. Weather assessments and decisions made

While all four of the severe weather phases generally show similar use of weather information according to the severe weather decision-making questionnaire, each phase had some unique assessments and associated decisions to be made. The prestorm and severe weather watch phases are characterized by communications with external entities and public organizations (i.e., schools, hospitals, etc.), primarily in preparation for a severe weather event.

Decisions and the assessments or conditions that lead to decisions made during the severe weather warning phase (as gathered from the questionnaire and scenario) are presented in Table 5. This phase also includes many communication decisions, but they are related to public outreach via the mass media and warning sirens. Another important feature of the decisions in this phase is the need to direct spotters and first responders into appropriate geographic areas in relation to the assessments made based on perceptual cues gathered from the information sources. Spotters are directed into areas that are deemed safe, but within clear view of the event. First responders are directed away from direct impact areas during an event, and then instructed afterward to attend to those areas where damage is believed to be the most severe.

The shifting decisions throughout the weather phases were reflected in the shifting values of the information sources seen in the severe weather decision-making questionnaire. For example, storm-relative velocity and composite reflectivity are continually rated as valuable, yet their median score increases with severe weather phase severity. These sources were primarily accessed in search of perceptual cues related to storm location and movement, which would aid in directing storm spotters and first responders as the storm approaches and moves through the area. Additionally, composite reflectivity includes algorithm output that may provide increased value to EMs with increasing weather severity.

Row 3 in Table 3 shows the total number of weather assessments reported for both experienced and novice EMs during the simulated weather scenario. Most EMs do not make weather assessments based solely on perceptual cues gathered from radar data without confirmation by local NWS forecasters, spotters, or other sources, to provide advice and “ground truth.” Also, as EMs make initial assessments, they may decide to cycle back and gather additional perceptual cues from various information sources. For example, one EM perceived that there was wind shear in a certain location based on radar data and made the assessment that there was potential for tornado development. This EM then decided to ask a spotter for additional cues about the presence of wind shear before solidifying their assessment and proceeding to make a decision to sound sirens. Most EMs also mentioned that they would communicate with local NWS forecasters before making a decision to sound a siren as well.

c. Potential emergency manager perception and assessment issues

Velocity interpretation continues to be a challenge to EMs. Many of the participants, particularly the novice EMs, were aware that despite having some training on interpreting velocity data, they could not accurately make assessments from this source. One EM mentioned difficulty understanding the velocity color palette and another had difficulty understanding the scale (thinking that negative values indicate very slow moving winds when negative values represent movement away from the radar). Additionally, one EM with <1 yr of experience was actually gathering incorrect perceptual cues from the information source (thinking that green pixels represented rainfall when green pixels represent movement toward the radar). Despite their lack of understanding, each novice EM also mentioned that they would communicate with local NWS forecasters to receive help in interpreting the velocity data.

Additionally, when some EMs tried to make weather assessments, their lack of understanding about the dependency of a radar’s physical location with respect to an approaching storm on velocity data became evident. Because current weather radars can only sense movement toward or away from the radar, when there were no high-velocity values, many EMs interpreted this perceptual cue to mean that there was no movement or wind shear at all.

d. Detailed analysis of experienced and novice emergency managers

The data in Table 3 were compared using a Mann–Whitney test. When comparing the behaviors of the experienced and novice EMs, there was a significant difference between the number of weather assessments made in the simulated scenario (W = 12.5, N1 = 4, N2 = 7, p = 0.019, one tailed) and a trend for a difference in the total number of cross validations (W = 15.5, N1 = 4, N2 = 7, p = 0.065, one tailed). However, while the median number of information sources used by novice EMs was less than the experienced EMs, there was no significant difference.

To further investigate the impact of experience, one senior EM (13 yr of experience) and one novice EM (<1 yr of experience) were selected for a more detailed analysis. During the scenario, the experienced EM made 14 total weather assessments, while the novice made only 3. The perceptual cues used by the novice EM were less detailed compared to the experienced EM. For example, the novice reported only the intensity and the movement of the storm, whereas the experienced EM’s reports included wind shear couplets, inflow and outflow regions, and the presence of a hook-echo structure—all more detailed perceptual cues used to make weather assessments.

There was no difference in the number of information sources used between the participants; however, there were frequency-of-use differences. The novice EM rarely consulted the velocity images compared to the experienced EM, and when these images were used by the novice, incorrect perceptual cues were gathered from the source. The novice EM was also less discriminating in weather source usage. This novice EM ranked 13 sources of information to be “very valuable” (score of 5) during severe weather warnings and yet only used or requested 5 of the 13 sources during the scenario. The experienced EM used or requested 4 out of 5 information sources that were scored as “very valuable.”

The experienced EM began the scenario with the decision to sound the sirens to warn the public. The EM made this decision after assessing the close proximity of the storm to the town in question combined with the assessment for a possible tornado. The novice EM ended the scenario without sounding any sirens. This EM said the decision not to sound the sirens was the result of not seeing a hook-echo structure in the radar reflectivity data. Both EMs mentioned that they would communicate with local NWS forecasters throughout their decision-making process.

4. Discussion

It was effective to use both questionnaires and a simulated weather scenario to gather information toward refining the EM descriptive decision-making model. This method has been previously used for investigating weather-related decision making (Hunter et al. 2003; Wiggins and O’Hare 1995). Although the analysis did not take place in the EMs’ natural environment, this technique afforded a repeatable and controlled method to study EM behavior. The questionnaires were useful in that participants had time to thoughtfully respond to questions regarding their information use and decision-making behavior. However, not all participants may have carefully and thoughtfully responded to each question. For this reason, the simulated scenario was also beneficial in that it provided a controlled and repeatable method to gather information about their job-related activities during a simulated weather event.

OK-FIRST instructors, NWS forecasters, and senior EMs provided a good starting point for the EM descriptive decision-making model. With respect to information sources, the initial model was mostly validated. The only information source that was not validated from the initial descriptive decision-making model (Table 2) was “warning decision updates” during the severe weather watch and severe weather warning phases. This product is issued by some local NWS weather forecasting offices, dependent on the weather, to provide EMs and media partners with more details about the local forecasters’ decision-making process. While we thought it would be used during the weather scenario, no EMs requested or mentioned this source during the scenario, so it was not included in the new model for the severe weather warning phase (Table 4). However, most EMs used the NWS text product provided and five EMs in the scenario mentioned that they would directly communicate with the local NWS weather forecasting office during severe weather by phone, radio, or teletype. Furthermore, one EM mentioned using National Oceanic and Atmospheric Administration (NOAA) Weather Radio to hear messages and warnings from the NWS. This is consistent with the findings of Morss and Ralph (2007) in that EM decision making is influenced by his or her confidence and trust in local NWS forecasters as an EM often wants to communicate with forecasters when gathering perceptual cues and developing weather assessments. With respect to decisions and/or actions during the severe weather warning phase, more actions were added that involved collaboration and communication with other organizations. Also, many of the decisions during a severe weather warning are made immediately based on the issuance of the NWS warning.

Not all information that is currently available to EMs during severe weather was provided during the scenarios. This may have affected which information sources were requested, as the sources were not in the forefront of their activities and they knew they could not use certain sources of information to make their assessments. Making use of an enhanced simulator tool that includes more information sources could produce more detail or nuance in information usage and associated weather assessment.

The perceived value of the information sources (Table 4) may not appear as diagnostic due to the list of sources used in the severe weather decision-making questionnaire. Further, it is not clear that the information sources EMs find to be most valuable are the best sources for weather-related decisions. It is possible that the most valuable sources of weather information to EMs are those that they understand better, have easier access to, or those that help them with less detailed perceptual cues, such as storm location and intensity (i.e., the big picture).

Results of this study cannot be completely generalized to operational settings because some aspects of an EM’s responsibilities during severe weather were excluded from the scenarios. Additionally, not all weather hazards and not every EM from Oklahoma were used to develop the descriptive decision-making model. The decision-making model is likely adaptable to other weather hazards, but the information sources used are likely a function of the details of the actual hazard. The varying education levels and the amount of OK-FIRST experience and training of the participants in this study also influenced the development of the model. More studies of EM information use and decision making should be conducted.

The conceptual framework of the EM decision-making process validated in this study is useful in visualizing how error and uncertainty in the beginning stages of the decision-making process (i.e., information sources and perceptual cues) propagate through and ultimately affect the decisions made. For example, when the novice EM made the error that green pixels on the screen was a perceptual cue for rain and not wind shear, he was later not likely to sound any sirens because there was no assessment that there was the potential for a tornado to form. It is possible that additional and correct perceptual cues could have led the novice EM to make a different weather assessment and associated decision. The perception and assessment issues experienced by EMs in the scenario represent areas that can be addressed by future training or alternate decision support tools.

Shifting perceptual cues, assessments, and decisions through the four severe weather phases could also have implications for training and decision support tool design. OK-FIRST system designers evidently recognized that the utility of the information sources is situation dependent, as the presentation of products is based on perceived utility in different situations (Morris et al. 2001). Furthermore, communication with storm spotters and local NWS forecasters is viewed as an important source for perceptual cue and weather assessment validation, particularly in the severe weather warning phase. Most novice EMs are also reliant on local NWS weather forecasters for assistance in interpreting radar velocity data due to their lack of experience. Research in this area could enhance the lines of communication between EMs and the NWS, potentially helping EMs make quicker and more accurate decisions that directly affect the public. Additionally, the amount of trust that EMs have in NWS forecasters and storm spotters could affect EM weather assessments and decisions, and this should be explored further.

Although future work should be conducted to make further validations and refinements to the descriptive model and conceptual decision-making framework, this effort will provide valuable information to researchers and practitioners concerned with emergency management decision making during severe weather. The results will be particularly useful for those involved with the CASA project as they attempt to integrate new information sources into the existing decision-making process of EMs in Oklahoma and future testbed sites.

Acknowledgments

This work was supported in part by the Engineering Research Centers Program of the National Science Foundation under NSF Cooperative Agreement EEC-0313747. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the National Science Foundation. The authors thank the emergency managers who participated in the study. They also thank three anonymous reviewers and Don Rude for comments on earlier versions of this manuscript.

REFERENCES

  • Baumgart, L., , and Bass E. J. , 2006: Presenting information in simulated real-time to support part-task weather scenarios. Proc. 2006 Int. Conf. on Systems, Man, and Cybernetics, Taipei, Taiwan, IEEE, 5209–5214.

  • Baumgart, L. A., , Bass E. J. , , Philips B. , , and Kloesel K. , 2006: Emergency management decision-making during severe weather. Proc. 50th Annual Meeting of the Human Factors and Ergonomics Society, San Francisco, CA, Human Factors and Ergonomics Society, 381–385.

    • Crossref
    • Export Citation
  • Brown, S., , Archer P. , , Kruger E. , , and Mallonee S. , 2002: Tornado-related deaths and injuries in Oklahoma due to the 3 May 1999 tornadoes. Wea. Forecasting, 17 , 343353.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Doswell C. A. III, , , Moller A. R. , , and Brooks H. E. , 1999: Storm spotting and public awareness since the first tornado forecasts of 1948. Wea. Forecasting, 14 , 544557.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • FEMA, cited. 2007: Tornado activity in the United States. Federal Emergency Management Agency. [Available online at http://www.fema.gov/plan/prevent/saferoom/tsfs02_torn_activity.shtm.].

  • Hunter, D. R., , Martinussen M. , , and Wiggins M. , 2003: Understanding how pilots make weather-related decisions. Int. J. Aviat. Psychol., 13 , 7387.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lusk, C. M., , Stewart T. R. , , Hammond K. R. , , and Potts R. J. , 1990: Judgment and decision making in dynamic tasks: The case of forecasting the microburst. Wea. Forecasting, 5 , 628639.

    • Search Google Scholar
    • Export Citation
  • McLaughlin, D., and Coauthors, 2007: Engineering Research Center for Collaborative Adaptive Sensing of the Atmosphere. Annual Report Year 4, Vols. I and II, NSF Cooperative Agreement EEC-0313747, University of Massachusetts—Amherst, 747 pp.

    • Search Google Scholar
    • Export Citation
  • Morris, D. A., , Crawford K. C. , , Kloesel K. A. , , and Wolfinbarger J. M. , 2001: OK-FIRST: A meteorological information system for public safety. Bull. Amer. Meteor. Soc., 82 , 19111915.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morris, D. A., , Crawford K. C. , , Kloesel K. A. , , and Kitch G. , 2002: OK-FIRST: An example of successful collaboration between the meteorological and emergency response communities on 3 May 1999. Wea. Forecasting, 17 , 567576.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morss, R. E., , and Ralph F. M. , 2007: Use of information by National Weather Service forecasters and emergency managers during CALJET and PACJET-2001. Wea. Forecasting, 22 , 539555.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • OCS, cited. 2007: OK-FIRST. Oklahoma Climatological Survey. http://okfirst.mesonet.org/.].

  • SPC, cited. 2007: Tornadoes and deaths by year and month, 1950–1999. NOAA/Storm Prediction Center. [Available online at http://www.spc.noaa.gov/archive/tornadoes/ustdbmy.html.].

  • Stewart, T. R., , and Lusk C. M. , 1994: Seven components of judgmental forecasting skill: Implications for research and the improvement of forecasts. J. Forecasting, 13 , 579599.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stewart, T. R., , Pielke R. Jr., , and Nath R. , 2004: Understanding user decision making and the value of improved forecasts. Bull. Amer. Meteor. Soc., 85 , 223235.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wiggins, M., , and O’Hare D. , 1995: Expertise in aeronautical weather-related decision making: A cross-sectional analysis of general aviation pilots. J. Appl. Exp. Psychol., 1 , 305320.

    • Crossref
    • Search Google Scholar
    • Export Citation

Fig. 1.
Fig. 1.

Conceptual framework of the decision-making process of EMs during a severe weather event. Each box represents a component of either the environmental system, the information system, or the perceptual and cognitive system of the EM. The solid arrows represent inputs from one system component to another and the dashed arrows represent influences between system components.

Citation: Weather and Forecasting 23, 6; 10.1175/2008WAF2007092.1

Fig. 2.
Fig. 2.

Tool for presenting weather information in simulated real time. The top two panels contain (left) base velocity and (right) base reflectivity data. NWS text information is on the bottom left panel and the legend is in the bottom right.

Citation: Weather and Forecasting 23, 6; 10.1175/2008WAF2007092.1

Fig. 3.
Fig. 3.

Number of EMs who used or requested different information sources during the simulated weather scenario.

Citation: Weather and Forecasting 23, 6; 10.1175/2008WAF2007092.1

Table 1.

Representative set of information sources available to EMs in the OK-FIRST program. These sources were assessed for their perceived value and frequency of use in the severe weather decision-making questionnaire.

Table 1.
Table 2.

Initial descriptive decision-making model of EMs based on interviews with instructors, NWS weather forecasters, and senior EMs in the OK-FIRST program.

Table 2.
Table 3.

Information use and weather assessment reports observed during the simulated weather scenario for both experienced and novice EMs.

Table 3.
Table 4.

Perceived value and use of information sources by emergency managers during a severe weather warning issued by the local NWS office.

Table 4.
Table 5.

Decisions made by emergency managers during a severe weather warning issued by the local NWS office and the associated assessments or conditions that lead to such decisions.

Table 5.
Save