Adding uncertainty information to hydrometeorological forecasts will not result in better disaster responses without additional measures.
Of all emergencies, 78% are weather related (Weaver et al. 2014), making hydrometeorological forecasts crucial for decision making in emergency management. However, forecasts have led to dangerous and expensive misunderstandings in the past. For example, in April 1997, emergency planners in Grand Forks, North Dakota, did not sufficiently prepare their city for an approaching flood because they did not correctly interpret the uncertainty inherent in the National Weather Service (NWS) outlook’s seasonal forecast. Even though the relative error in the NWS outlook turned out to be smaller than the average relative error, the flood damage in Grand Forks amounted to over $1 billion (U.S. dollars), with much of the blame falling on the NWS. Pielke (1999) called the 1997 Grand Forks case a “misuse” of NWS forecasts. Likewise, several other scholars have described the relationship between river forecasts and decision making as weak (e.g., Hunemuller 2010; Rayner et al. 2005; Penning-Rowsell et al. 2000; Golden and Adams 2000; Parker and Handmer 1998).
This paper describes emergency managers (EMs), a large and important group of NWS clients, and their use of hydrometeorological forecasts in decision making, with a special focus on the ways EMs cope with forecast uncertainty. This paper arrives at a number of recommendations for the NWS (printed in boldface) to increase the value of hydrometeorological forecasts for EMs.
Although short-term river forecasts (Fig. 1) serve as the main focus of this paper, they are used as a basis for drawing general conclusions regarding the communication of NWS-communicated forecast information.1 The 13 river forecast centers (RFCs) of the NWS perform a large-scale, technically sophisticated effort to publish river forecasts for approximately 4,000 river gauges in the United States. The National Hydrologic Warning Council (2002) has conservatively estimated that these short-term forecasts reduce average annual flood losses by $433 million (U.S. dollars; 2000 cost levels, excluding saved lives), which is 10% of the aggregate flood damage.
Example of a National Weather Service river stage forecast. Forecast for Monogahela River at Elizabeth Lock and Dam on 5 Dec 2013. The thick line represents the observed river stage height in the past few days. The dotted line represents the river forecast in 6-h intervals for the next few days. Action, minor flood, and moderate flood stage indicate the different degrees of calamity. (Available online at http://water.weather.gov/ahps2/hydrograph.php?wfo=pbz&gage=ELZP1.)
Citation: Bulletin of the American Meteorological Society 97, 1; 10.1175/BAMS-D-13-00275.1
Interviewees.
This paper is based on interviews with EMs2 and a review of the literature. We contacted 45 EMs in towns along rivers. A special effort was made to reach local EMs with hands-on experience. We conducted 17 in-person interviews with EMs in Pennsylvania (7), Oklahoma (7), and Arkansas (3).3 The interviews averaged 50 minutes and were semistructured, utilizing prepared questions on training, daily routine, emergency operation, hydrometeorological forecasts, and forecast uncertainty.
Of the 17 interviewed EMs, 3 were female, and overall the EMs had an average work experience of approximately 12 years. Four interviewees were not paid for their efforts as an EM, while seven were full-time EMs; for the remaining six, emergency management was an extra hat that they wear as part of their full-time profession. There were 15 EMs responsible for a town or borough, and 2 were responsible for a county. Finally, regarding familiarity with hydrometeorological forecasts, 16 of the 17 interviewed EMs knew the NWS website and how to access river forecasts through various channels, 15 interviewees made use of these forecasts, and 2 had trouble describing them.
The interviews give an impression of the reality on the ground but are not representative of the entire United States nor all types of severe weather. For example, the experiences of EMs presented here are likely to be very different from those in jurisdictions along the Mississippi, where floods can be anticipated weeks in advance. The findings of this paper also do not pertain to flash floods in watersheds with very short hydrologic response times. Throughout the paper, quotes (presented as extracts) illustrate the observations and arguments made.
THE PROFESSION OF EMERGENCY MANAGEMENT.
In the United States, there are thousands of EMs engaged in public service. Most states require or otherwise encourage the appointment of an EM for each political subdivision, such as a municipality or county (e.g., Pennsylvania and Arkansas) or each incorporated jurisdiction (e.g., Oklahoma) (Pennsylvania’s Emergency Management Services Code, 35 Pennsylvania C.S. Section 7501; Arkansas Code 12-75-118; Oklahoma statues—Title 63 Public Health and Safety Section 63-683.11A). Accordingly, each state has hundreds or thousands of EMs in the public sector (Table 1), with many more working for companies, schools, hospitals, and so on.
Number of counties and municipalities as a proxy for the number of EMs in each state. In most states, each jurisdiction or at least county is required to appoint an EM.
Traditionally, a (retiring) police or fire chief is named the local emergency coordinator. Subsequently, police and firefighting professions have had a considerable influence on emergency management. In our sample, seven EMs had a background as a firefighter, police officer, or paramedic. The other EMs came from a wide variety of previous professions. Our sample contained a librarian, coal miner, oil worker, veterinarian, military officer, industrial safety engineer, and banker. Three EMs in our sample had a college degree in emergency management, which may be unrepresentative of the future, as there is a trend to professionalize full-time emergency management. More colleges are offering emergency management degrees, motivated in part by Federal Emergency Management Agency (FEMA)’s Emergency Management Higher Education Program established in 1994.4
Tasks/core competencies.
One thing to understand about emergency management: it is more an idea than it is a fact. If you tell me you are a firefighter out in California, I generally know what you do. If you say, I am a cop in Arizona, I generally know what you do. If you tell me, I’m an emergency manager from Iowa, I know you probably have something with communications; you definitely have something with plans. But what else you do…
The quote above illustrates that emergency management is not a clearly defined profession. As another interviewed EM put it, EMs are tasked with everything that “does not fit nicely into the fire department or EMS [emergency medical services].” Therefore, because EMs each operate in different organizational structures, task allocation varies across jurisdictions.
Just before or during severe weather events, when hydrometeorological forecasts are being used, EMs mainly coordinate and improvise, as the following two quotes describe:
You know, if the police comes to the scene they bring guns, firemen bring fire trucks; the emergency manager brings a phonebook.
You need to get over there in a boat…I don’t have a boat, but I will find a boat.
Additionally, EMs’ resources to prepare and respond to severe weather events vary wildly. On the one end, one interviewed EM’s only resource was his phone, forcing him to also spend $2,000 of his own money each year. On the other end, an EM a few miles away had his own fortified office, staff, jeeps, trailers, and boats.
The preparations for severe weather events, the responses to them, and the entailing decisions depend on the tasks and resources of each EM. These resources vary greatly between jurisdictions, making EMs a very diverse group of NWS customers.
Qualifications.
Using hydrometeorological forecasts for decision making is a learned skill. Every state decides on its own set of qualifications that a candidate must meet in order to become an EM. The only overview of EM qualifications that we are aware of dates from 1990 [International Association of Emergency Managers (IAEM) 1990], which is from before the paradigm-shifting events of 9/11 and Hurricane Katrina.5
Generally, the immediate job requirements for becoming a paid or volunteer EM are minimal. Usually, as in the case of Oklahoma, an aspiring EM must have U.S. citizenship, a high school diploma or equivalent, a valid driver license, and a social security number (63 O.S. section 63-683.11B). Some states require work experience in a relevant field or, as in the case of Pennsylvania, professional competence and the capability of planning and coordinating (35 Pennyslvania C.S. section 7502d).
Once a candidate has become an EM, most states require initial training within 1–3 years of their appointment, typically by enrolling in courses offered by FEMA’s training facility, the Emergency Management Institute (EMI). Additionally, EMs must regularly attend conferences and workshops. Some states encourage emergency management education rather than requiring it. Alabama, for example, does not require EMs to attend any classes or trainings but places a financial incentive on following a very thorough certification program. For the NWS, it is interesting to note that only 31% of training courses are on weather-related topics, while 78% of all incidents are weather related (Weaver et al. 2014).6 Since the National Weather Service Community Preparedness Program became part of FEMA in 1979, the NWS seems to be rarely directly involved with the courses at EMI. The 2014 course catalog lists only one course that is cotaught by NWS staff (FEMA 2014).7
HYDROMETEOROLOGICAL FORECASTS IN EMERGENCY MANAGEMENT.
Use of river and weather forecasts in emergency management.
While EMs use hydrometeorological forecasts for their own decisions (e.g., whether to activate the emergency operations center, to open shelters, or to relocate people), they also provide forecast information and guidance to a number of stakeholders. These stakeholders, in turn, prepare for and respond to severe weather events with or without coordination of the EM. For example, depending on the jurisdiction, roads may be closed by the Public Works department, fire or police chiefs may coordinate rescuing activities, and the mayor may sign off on evacuations. Citizens are responsible for their own property, and often even evacuation decisions are left as optional to individuals. Ultimately, every person and business is responsible for their own safety, as the following two quotes describe:
I call them [at the power plant], we let them listen to the briefing, and they make their own decision based on the information they are getting.
Would I tell everybody to evacuate? No, I would not. I would say, here is the information, and it would probably come across very not nice…Make your own [#!°&%] decision.
To enable people to take care of themselves, effective dissemination of hydrometeorological forecasts is one of an EM’s most important tasks, especially in the preparation phase just before a severe weather event. Three out of nine standard preparation tasks identified by the International Association of Emergency Managers involve gathering or disseminating weather information (see Table 2).
Emergency managers’ tasks before and after a severe weather event. Source is IAEM (2014). Tasks involving gathering and disseminating weather forecasts are in italics.
Emergency managers as part of the dissemination process.
In terms of dissemination, the EM has three main tasks: The EM alerts decision makers to the situation, interprets the hydrometeorological forecasts, and motivates decision makers to take action. Each will be discussed in detail below.
Alerting people
First, EMs constantly keep an eye on the weather and river situation and alert decision makers (i.e., department heads, police and fire chiefs, city officials, and citizens) if they think a critical situation is approaching. The following three quotes describe the EMs’ significant contribution to forecast dissemination:
We would be the receptor of…information that comes in…We bring all the decision makers, city manager, public works director, fire, police and have them coordinate and get them working as a team.
I will make sure it is in the newspaper. I will make sure it is on the radio. I will make sure that TV can tell people.
We rely on every type of social media you can. Peer notification is a major player.
We try to use the radio, we are starting to use a little bit of social media. It is a real struggle getting that through the City, to agree to social media, but it is going to be effective.
To alert every person at risk, EMs must navigate various obstacles that the NWS itself would not be able to overcome. For example, as illustrated by the three quotes below, EMs have to overcome long chains of communication that include key decision makers and minorities. This underlines the importance of the EMs as messengers of the NWS.
Long chain of communication:
We go to [our] boss, who is the public safety director and we say, “This is the information; what do you want us to do?” [The public safety director] is the one that is final for anything that is major…because he goes and talks to the mayor.
Unsuccessful communication to key decision makers, in this case to the mayor, who often is an influential actor in the emergency operations center:
All you could see was a window of the attic, where these people had fled to and some were on the roof. 100 some people. One of them was our mayor.
Difficulty reaching minorities:
25% of my population would be considered Spanish speaking. And the radio and TV stations are now carrying both Mexican-speaking channels. But they don’t give local weather…I have no circulated Spanish paper to put the information in.
Given these obstacles, the NWS must ensure that its weather and river forecasts are formatted to assure that its information is not diminished, distorted, or made inaccurate as it is communicated and recommunicated over a variety of channels, from the forecasting center, to the local forecast office, to the emergency manager, and finally to the decision maker.
Further, the objective of formatting forecasts should not only be correct understanding by the users but also correct dissemination. In an ideal system, EMs would convey risks to the public in nontechnical terms (Perry and Nigg 1985). Therefore, not only must EMs understand the technical information in forecasts, but they must also translate forecasts into nontechnical language. This task would become even more challenging if uncertainty information was included in forecasts. Therefore, it is crucial that EMs learn how to communicate uncertainty to the general public through FEMA courses such as “G-272 Warning Coordination” and “G-289 Public Information Officer Awareness.”
Interpreting Hydrometeorological Forecasts
Second, EMs add local information to the hydrometeorological forecasts that the NWS cannot provide. EMs often know their jurisdictions well enough to translate forecasted river water levels into projections of which places will flood. Additionally, many EMs have to interpolate between river gauges because there is no gauge close to their jurisdiction. The quotes below show two examples of assessing the situation:
When the foam is on the middle of the river, it is coming up….If I see big trees coming down, I know it is a lot water. If I don’t see big trees, well, it is not a lot [of] water.
[If River A] isn’t up and the [River B] is down…[and the gates at the] Lake [are] closed, then I think it is going to go away. Or I tell them, see [River A] is continuing to flood, it crests up there, 12 hours later it crests down here. Same for [River B]…If I get both crests here at the same time, I have a problem. If this one is gone before this one gets there, then there is not a lot of current.
Motivating People To Take Action
Third, the EMs motivate people to take action and suggest actions to take. An interviewed EM stated:
We may make phone calls to certain hospitals and ballparks and things like that. “This is the time you probably need to pull people off the field,” things like that.
The possibility of false positives—a consequence of uncertainty—is a major reason why EMs often have a hard time encouraging people to take precautionary action. One EM described it as follows:
They have heard [a severe weather warning] so much, that…they are expecting me as a government official to take them by the hand and take them to the cellar when they need to go to the cellar.
In this context, it is important that EMs enjoy authority, especially in closely knit communities. Through their profession and mindset, EMs often have a strong, dense personal network. For example, in Tornado Alley, EMs and their communities have been through many critical situations together, creating a high degree of respect for the EMs. The following quotes illustrate how EMs use their reputation to motivate people to take action:
I just call. “Hey, we are going to flash flood.”…That is all I have to say to them. That is called the personal touch.
What I normally tell them is…“I want to get a good look of you before I leave, because later on when I am called to identify you, I want to know who you are.” And most times they leave when you tell them that.
In short, through connections with the people in their communities exposed to flooding, EMs have the ability to prevent decision makers from becoming paralyzed when faced with uncertainty.
Factors limiting the value of hydrometeorological forecasts.
Three characteristics of river and weather forecast undermine the value of forecasts for EMs.
First, because of the data sparseness inherent in extreme events, forecast uncertainty and forecast errors for emergency-level events can be substantial. For example, forecast errors of several feet are not uncommon for short-term river forecasts (Welles et al. 2007; Pielke 1999), which has been the case for decades (Welles et al. 2007).
Second, many hydrometeorological forecasts—both deterministic and probabilistic8—do not routinely include information such as an uncertainty range, an indication of historical average prediction error, or any other measure of uncertainty. Of particular relevance for this study, short-term river forecasts (Fig. 1) that are used in emergency management decision making do not include such uncertainty metrics yet.
Third, it is extremely difficult for river and weather forecast users to estimate the expected forecast error themselves. Because of the infrequency of extreme events, most people cannot draw on experience to quantify prediction errors. Additionally, forecast uncertainty is complex. Forecasts with greater lead time are more uncertain, and extreme events tend to have larger forecast errors than more common ones. In the case of river forecasts, decision makers do not seem to be aware that forecasts predicting high river stages—when forecasts are needed most—are subject to more uncertainty than forecasts predicting low river stages (Morss and Wahl 2007).
These three characteristics limit the usefulness of weather and river forecasts and consequently reduce their use in emergency management.
Strategies to cope with forecast uncertainty.
The three characteristics that limit the value of hydrometeorological forecasts for EMs, described above, are all related to forecast uncertainty. The interviewed EMs were very well aware of the uncertainty in river forecasts. Through the daily use of rain and temperature forecasts, they have often experienced the general limits of hydrometeorological forecasts.9 EMs with flood experience also encountered different forms and degrees of uncertainty in river forecasts first hand, as the following quotes show:
The river forecasts are unpredictable…Sometimes they are close, sometimes they are way off.
During the 2007 flood, their first forecast was 19 feet. We went 10 feet above that.
There have been a couple of times, when I called [NWS] and said, “My river is on flood stage and I don’t see nothing on the website about it.”
They usually overpredict; they usually predict it to be higher than it ever reaches.
Even though the EMs are aware of the uncertainty in river forecasts and can explain its causes, none of the interviewed EMs—not even those with flood experience—were able or willing to quantify a representative forecast error:
I couldn’t tell you…It does vary a little bit. But 1 to 5 feet difference, I couldn’t tell you.
When floods occur, EMs gain experience to cope with uncertainty by trial and error, incurring potentially unnecessary damages, as the following quote illustrates:
What I wasn’t doing, and that is kind of my own fault, thinking, yeah, it is pouring rain here, but what is it doing up above?
Even though they might not explicitly be aware of it, the interviewed EMs use a number of strategies to cope with the uncertainty in hydrometeorological forecasts, described in the following section.
Extensive Information Collection To Make Own Estimate
The interviewed EMs appreciated receiving any information concerning approaching weather events or floods. Especially in small towns, it seemed as if many of the EMs soak in all of the information they could possibly receive to get an idea of what was going on in their community, even when it did not directly relate to the decisions at hand. In their mind, the weather situation is an integral part of the state that their community is in and cannot be separated from other ongoing activities. For example, one EM appreciated being notified that airplanes at a nearby airport were being deiced even though the decision at hand was whether the local Santa Claus party should take place given the winter weather. This reflects the EMs’ attitude toward their job: they tend to care for a community as a whole rather than just ensuring safety. While not all information gathered by EMs is necessarily relevant, the value of gathering additional information may be greater than anticipated; Morss (2010) observes that the most important decisions in successful responses to flooding were decisions to gather more information, for example, using monitoring crews.
EMs retrieve weather information from a wide variety of sources: NWS publications, personnel, local media, and their personal networks. One EM mentioned the importance of his personal network to find out what was currently happening:
I have an uncle who lives 30 miles west. He will call or somebody else’s brother-in-law lives up north 60 miles and he will call “We have 6 inches of rain.” …So normally, through the channels of families and business…the information trickles in from one way or the other.
Exploiting personal networks for information is not exclusive to EMs; other NWS clients employ the same strategy. For example, Rayner et al. (2005) tells of a water supply operations manager who relies on a brother-in-law 60 miles away for information about weather conditions.
Where this is not yet the case, the NWS and state emergency management associations (EMAs) should (jointly) offer hotlines, chat rooms, or mentoring networks where EMs can discuss the correct interpretation of hydrometeorological forecasts. All weather and river forecasts have room for interpretation because of the inherent uncertainty in forecasting. If EMs cannot reach an expert when they face uncertainty (e.g., because of jammed phone lines), they are likely to acquire additional information through more ad hoc sources, which may be of lesser quality.
In interviews, 9 out of 17 EMs explicitly mentioned how much they appreciated having a close relationship with NWS personnel. An interviewed EM stated:
Because we have that relationship I am able to pick up the phone and call them. And they won’t talk down to me…I make them look good, they make me look good; it is a partnership. And it has been wonderful when we can work together.
Using their local knowledge, EMs make their own interpretation of the information they have collected. One EM describes his thought process as follows:
I would have…to see what is going on on the radar and see where the rain is falling to say…I give [NWS] 75% probability that [the forecasted water level] is going to happen.
Although the NWS forecast is only one piece in an EM’s information puzzle, because it is the official interpretation of the situation at hand, it is a very important piece. The NWS forecast serves the EMs as a standard to build on and to compare their own estimates to. Nonetheless, EMs, particularly those with less experience, refrain from making uncertainty estimates. One EM stated:
But try to analyze [the forecast]…that is not what our job is.
To prevent (inexperienced) EMs from ignoring uncertainty, uncertainty information, such as the historical forecast error or at least a prominent note that the uncertainty exists, should be published alongside river and weather forecasts. To facilitate EMs in making their own estimate, both deterministic and probabilistic forecasts of any format need a description of the main sources of uncertainty (e.g., temperature development with regard to snowmelt, dam gate operations, or expected rainfall upstream). This way, it will be easier for EMs to combine the weather and river forecasts with other information that they use to produce their own estimates.
Using Observed Conditions Rather Than Forecasts
As another mechanism to cope with forecast uncertainty, EMs try to use the hydrometeorological information that has the least uncertainty: observational data. For example, rather than the river forecasts, EMs often use a combination of observed stage heights and weather radar. If the river water levels are high or rising, and the radar indicates that more rain is to be expected upstream, EMs know that they are likely to experience flood problems. Further, the river forecast is updated too irregularly, and the forecasted crest levels vary too much to form the basis of decisions. Additionally, the forecast’s uncertainty is less intuitive than that of the weather radar. The quotes below illustrate the discussion above:
When there is a flood forecast, certainly we are on alert. But I spend as much time driving down to the creek as I do listen to the forecast.
The forecast can change normally every hour. Sometimes during an event, it may change every half hour.
The NWS puts out their temperature forecast at 9:45 AM and 4:45 PM…The river forecast people put up forecasts whenever they want to. There is no set time. So I tell my [town’s] people…that we are expecting flooding. “Stay tuned to the NOAA weather radio.” …I don’t have the latest information but you can stay updated, because I might be gone for an hour or two on a delivery and they put out a new forecast and I am not aware of it.
These findings are consistent with Rayner et al.’s (2005) observation that water resource managers tend to rely heavily on observed information from monitoring groups while largely ignoring model results.
When developing new forecast formats, the NWS should consider which format would make weather and river forecasts more salient to decision making than they are today. Communicating forecast uncertainty does not reduce it. Therefore, it seems possible that EMs would still prefer using weather radar and observational weather data over forecasts, even if the latter included uncertainty information. Radar and observational weather data will always be inherently less uncertain than forecasts, regardless of a forecast’s format. The NWS should question how weather and river forecasts can be formatted to increase their practical value. For example, offering a description of the situation (e.g., how much more rain would have to fall upstream to cause serious flooding) and the sources of uncertainty would make flood forecasts more salient for EMs, thus enabling them to more accurately make their own estimates of the situation.
Not Using Lead Time To Take Preventive Measures
As the next few quotes indicate, the uncommunicated uncertainty in river forecasts, in regard to providing lead time and identifying the crest level to prepare for, severely limits their value. All of the interviewed EMs stated that they normally start taking substantial preventive measures only when they (or their monitoring crew) see the water rising with their own eyes. Two EMs said:
…until the water actually comes and you know which way it’s going to go and what floods, you cannot take specific measures.
…with the topography of the area, with the hills and valleys, lead time really…you cannot rely on [the lead time].
The quotes suggest that EMs make limited use of the lead time provided by river forecasts because of forecast uncertainty.
In the same vein, many interviewed EMs did not attach great value to having more lead time. The opinions of two interviewed EMs were as follows:
I’m not sure if [more lead time] would change much to be truthful.
People that did not know what to do died [in that flood]….So the biggest part of these things is [by the time] the fire guys get to your house, you may be dead.
Effectively, the latter quote is saying that greater lead time for emergency responders would not save lives. A possible explanation is that policemen, firemen, and paramedics place a low value on lead time because they are used to reacting to rather than preparing for emergencies. Another explanation could be that EMs have not been trained to translate uncertain hydrometeorological forecasts into preventive action. Participants of the OKFirst program were a notable exception. Through this program, EMs received training and real-time weather data for weather-related decisions from the Oklahoma Climatological Survey (OCS) and Oklahoma Mesonet. Asked about their OKFirst experience, participating EMs were appreciative of being taught not only how to read but also how to interpret weather radar information. For example, this training allowed them to more reliably decide whether it was justified to cancel a ball game, something that they had not previously received instructions on before OKFirst.
To make better use of the lead time, state EMAs should ensure that every EM receives basic training in the use of hydrometeorological forecasts in emergency response. Additionally, the NWS should ensure that interpreting the uncertainty in deterministic and probabilistic forecasts is taught in an applied, “boots on the ground” manner, for example, by developing course material or coteaching courses.
Selective Communication of Forecast Uncertainty
As mentioned earlier, most interviewed EMs make their own estimates of the situations they are faced with using weather and river forecasts as benchmarks. While the interviewed EMs share their assessments and concerns regarding NWS forecasts with the other people in their emergency operations center, they will only repeat the official published forecast information to the citizens. The following quotes demonstrate this difference between internal and external communication:
Internal communication to people in the emergency operations center:
So I put kind of all together and then I have another briefing [for the staff]. “Guys, this is the information I have. It appears as if it is going to come up another couple of feet. But it might not.” That is a guessing game for floods.
Last thing that you want is that your boss, the city manager, is being caught off guard…And so it is like if I know this and I give it to you, then you know it, too. So it is accountability.
External communication to citizens and businesses:
Whatever they [at NWS] tell me, I take what they say and act accordingly. I don’t take a chance of saying maybe it’s wrong. I can’t do that.
If I start putting out my own forecasts, if I start telling people what I think…and then I’m wrong. And I…is the city responsible or liable?
Interviewer: Do you tell them, “The forecast is 19 feet, but in the past it’s been 10 feet above that?” Interviewee: “I’m not going to. No…Even though I know…The less I say, the better off I am, because the media will come to haunt me.”
There are three possible explanations of why EMs are hesitant to share their assessment of the situation with the public: They might not want to risk liability claims for themselves or their employer; they might fear that they will be held accountable, for example, by the media or by their superiors; or they might not feel sufficiently confident about their judgment.
The logic for this type of hesitation is that the EMs feel responsible if their interpretation of the forecast causes any damage. Consequently, the EMs prefer to only pass on the official information.
When asked, most EMs did not know what legal consequences they might face, but as the above quotes illustrate, they were concerned. In describing lawsuits against weather forecasters, Klein and Pielke (2002a) explain that the Federal Tort Claim Act (FTCA) protects the federal government against such claims because forecasting falls under the FTCA discretionary function exception. Most states have a similar immunity statute (Swanson 2000).
We are aware of only one lawsuit where a county EM was sued in connection with weather forecasts. In 1989, 30 students were injured or killed when a wall at an elementary school collapsed because of a tornado and strong winds. The relatives of the deceased sued, among others, the county for failing to warn the school, even though the NWS had published a tornado watch (not warning). In 1992, the Appellate Division of the Supreme Court of the State of New York ruled in Litchhult v. Reiss (New York Appellate Division 1992, p. 1069) that the county was not liable because “the County had to exercise its discretion in determining whether evacuation or other extreme measures would be taken in a similar manner as if a tornado warning had been issued.”
The case history thus does not give the EMs much reason to worry about liability. However, Klein and Pielke (2002b, p. 1806) warn that there is a large gray area in cases when “the forecast may have been made in good faith, [but] it strayed from established professional standards” that define reasonable care. As long as professional standards regarding the dissemination of weather and river forecasts have not been defined for EMs, EMs will remain reluctant to voice their own assessment of the situation in public. Additionally, it is important that the NWS and FEMA together evaluate the legal aspects of using and interpreting hydrometeorological forecasts. Most importantly, EMs need to be informed about their legal situation to reduce their current uncertainty.10
EMs and the NWS would benefit if the NWS published its uncertainty information as an integrated part of their weather and river forecasts. Not only would that decrease the (already low) probability of liability claims, if uncertainty information was published by a federal authority such as the NWS, EMs would feel more confident—in a legal sense—to communicate the uncertainty to decision makers.
ADDITIONAL DISCUSSION.
Zooming out, the uncertainty in weather and river forecasts is not the uncertainty that EMs are most concerned about. Cascading events—such as clogged stormwater drainages from debris, medical needs of evacuees, hazmat spills, river traffic (recreational boating and loaded barges), drinking water and utility outages, and snakes—cause EMs the worst surprises:
Usually, what is unexpected is when the infrastructure fails or collapses.
It sounds stupid, but [lack of air conditioning] is a thing that can make a small emergency a big emergency.
Morss (2010, p. 91) describes a particularly turbulent situation: “In Fort Collins, in the trailer park area alone, emergency responders had to handle a water rise of 1.7 m (5 ft) in 3 min, three trailers on fire, 162 people in imminent danger requiring rescue, a building explosion, and derailment of a train that included a car carrying chlorine gas, with no advance notice.” The comparatively small forecast uncertainty is often addressed by working with safety margins, such as an extra foot of freeboard for flood defenses (Morss et al. 2005).
While we have seen in this paper that communicating uncertainty could be beneficial, there are limits to this finding. For example, an EM usually does not have the time to incorporate complex information about a relatively small uncertainty into his/her decision-making process (Morss et al. 2005). Additionally, uncertainty information can create confusion, increase reliance on experience, or trigger a “wait and see” strategy, effectively delaying preventive action (Demeritt et al. 2007).
In sum, most EMs have found ways to cope with forecast uncertainty and do not perceive it to be a major problem. Instead, the NWS should give more attention to what other problems forecast uncertainty causes, such as the liability and accountability concerns discussed above. In those cases, communicating uncertainty will have much more value than using uncertainty estimates to formally optimize emergency responses (e.g., Verkade and Werner 2011).
CONCLUSIONS.
The value of uncertain weather and river forecasts for emergency managers can be increased in a number of ways. As a first step, it is important to realize that interpreting weather and river information is only one out of innumerous skills that EMs have to master. Additionally, emergency management was not the main profession of most interviewed EMs; in small towns, they often were volunteers. Nonetheless, EMs play an important role in the dissemination of hydrometeorological forecasts. EMs bring forecasts to the attention of decision makers, often interpret it for them, and motivate them to take action.
This study clearly shows that the NWS is taking the right steps by advancing probabilistic forecasts and including other types of uncertainty metrics. Without this information, people on the ground are unlikely to learn about the uncertainties. But as is already the case with deterministic forecasts, simply providing information to decision makers does not automatically result in adequate decisions in emergency management. There are a number of things the NWS, in cooperation with FEMA and state EMAs, can do to significantly increase the utility of its hydrometeorological forecast products:
Consider what impact publishing uncertainty information will have on the role of a river or weather forecast in an emergency response. For example, currently, the use of river forecasts for decision making is limited, mainly because of potentially large, uncommunicated forecast errors. However, these forecasts do play an important role in liability and accountability issues because the forecasts are regarded as the official interpretation of the situation.
Clarify the impact of hydrometeorological forecasts on liability considerations of local EMs. Potentially some emergency managers would reassess their accountability concerns if uncertainty information was an integral part of river and weather forecasts, that is, they would feel more confident to communicate uncertainty to the public.
Format weather and river forecasts so that people not only understand the forecast but also share the information with others correctly. Sharing information on uncertainty is likely to be more difficult than sharing a single-point forecast.
Provide personal assistance in cases where EMs have trouble interpreting uncertainties. If experts are not available for clarification of the hydrometeorological forecast when faced with uncertainty, this gap will undoubtedly be filled in by other, more ad hoc sources of information.
Amend EMs’ trainings, workshops, and conference presentations on NWS products with explanations on how to utilize their products in crisis situations. Additionally, provide training on how to make decisions under uncertainty and how to communicate uncertainties to the public. Otherwise, uncertainty can lead to inaction. Interactive small-scale case studies, such as postponing a local baseball game because of inclement weather, seem to work best.
ACKNOWLEDGMENTS
Most importantly, many thanks go to the emergency managers who made the interviews possible. Additionally, we are grateful to the reviewers for their constructive input. Finally, Gabe Chan has done a tremendous job polishing the language of this article. Thank you! As to funding, Frauke Hoss was supported by an ERP fellowship of the German National Academic Foundation and Climate and Energy Decision Making (SES-0949710) through a cooperative agreement between the National Science Foundation and Carnegie Mellon University (CMU).
REFERENCES
Arkansas Department of Emergency Management, 2014: Training plan: January 2014 update. ADEM Training Plan Rep. V2.5 Rv. Nov 2006, 58 pp.
Demargne, J., and Coauthors, 2014: The science of NOAA’s operational Hydrologic Ensemble Forecast Service. Bull. Amer. Meteor. Soc., 95, 79–98, doi:10.1175/BAMS-D-12-00081.1.
Demeritt, D., H. Cloke, F. Pappenberger, J. Thielen, J. Bartholmes, and M. H. Ramos, 2007: Ensemble predictions and perceptions of risk, uncertainty, and error in flood forecasting. Environ. Hazards, 7, 115–127, doi:10.1016/j.envhaz.2007.05.001.
FEMA, 2014: Fiscal year 2014 training catalog: Train, exercise, educate. Emergency Management Institute Rep. [Available online at http://training.fema.gov.]
Golden, J. H., and C. R. Adams, 2000: The tornado problem: Forecast, warning, and response. Nat. Hazards Rev., 1, 107–118, doi:10.1061/(ASCE)1527-6988(2000)1:2(107).
Hunemuller, T. J., 2010: Review and analysis of the National Weather Service river forecasts for the June 2008 eastern Iowa floods. Ph.D. thesis, University of Iowa.
IAEM, 1990: Existing standards for local emergency management coordinators. International Association of Emergency Managers Rep.
IAEM, 2014: Emergency program manager: Knowledges, skills, and abilities. FEMA, accessed 9 October 2014. [Available online at http://training.fema.gov/EMIWeb/edu/EMCompetencies.asp.]
Klein, R., and R. A. Pielke Jr., 2002a: Bad weather? Then sue the weatherman! Part I: Legal liability for public sector forecasts. Bull. Amer. Meteor. Soc., 83, 1791–1799, doi:10.1175/BAMS-83-12-1791.
Klein, R., and R. A. Pielke Jr., 2002b: Bad weather? Then sue the weatherman! Part II: Legal liability for private sector forecasts. Bull. Amer. Meteor. Soc., 83, 1801–1807, doi:10.1175/BAMS-83-12-1801.
Morss, R. E., 2010: Interactions among flood predictions, decisions, and outcomes: Synthesis of three cases. Nat. Hazards Rev., 11, 83–96, doi:10.1061/(ASCE)NH.1527-6996.0000011.
Morss, R. E., and E. Wahl, 2007: An ethical analysis of hydrometeorological prediction and decision making: The case of the 1997 Red River flood. Environ. Hazards, 7, 342–352, doi:10.1016/j.envhaz.2007.09.004.
Morss, R. E., O. V. Wilhelmi, M. W. Downton, and E. Gruntfest, 2005: Flood risk, uncertainty, and scientific information for decision making: Lessons from an interdisciplinary project. Bull. Amer. Meteor. Soc., 86, 1593–1601, doi:10.1175/BAMS-86-11-1593.
Morss, R. E., J. Demuth, and J. K. Lazo, 2008: Communicating uncertainty in weather forecasts: A survey of the U.S. public. Wea. Forecasting, 23, 974–991, doi:10.1175/2008WAF2007088.1.
Morss, R. E., J. K. Lazo, and J. L. Demuth, 2010: Examining the use of weather forecasts in decision scenarios: Results from a US survey with implications for uncertainty communication. Meteor. Appl., 17, 149–162, doi:doi:10.1002/met.196.
National Hydrologic Warning Council, 2002: Use and benefits of the National Weather Service river and flood forecasts. EASPE, accessed 19 February 2014. [Available online at www.nws.noaa.gov/oh/ahps/AHPS%20Benefits.pdf.]
National Research Council, 2006: Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. National Academies Press, 112 pp. [Available online at www.nap.edu/catalog.php?record_id=11699&;utm_expid=4418042–5.krRTDpXJQISoXLpdo-1Ynw.0.]
New York Appellate Division, 1992: Lichult v. Reiss, 183 A.D.2d 1067. Accessed 4 March 2014. [Available online at https://scholar.google.com/scholar_case?case=9093875182234136223&q=weather+%22emergency+manager%22&hl=en&as_sdt=6,39.]
Nicholson, W. C., 2006: Chapter 14: Legal issues in emergency management. Fundamentals of Emergency Management, M. K. Lindell, C. S. Prater, and R. W. Perry, Eds., Federal Emergency Management Organization, 392–424. [Available online at http://training.fema.gov/EMIWeb/edu/fem.asp.]
NOAA/National Weather Service, 2011: NWS river forecasting. Accessed 24 January 2014. [Available online at www.crh.noaa.gov/images/dvn/downloads/backgrounder_DVN_River_Forecasting.pdf.]
Oklahoma Department of Emergency Management, 2009: Training requirements: Newly appointed Oklahoma emergency management directors. Accessed 10 February 2014. [Available online at www.ok.gov/OEM/Programs_&;_Services/Training_Information/2009newdirector.html.]
Parker, D. J., and J. W. Handmer, 1998: The role of unofficial flood warning systems. J. Contingencies Crisis Manage., 6, 45–60, doi:10.1111/1468-5973.00067.
Penning-Rowsell, E. C., S. M. Tunstall, S. M. Tapsell, and D. J. Parker, 2000: The benefits of flood warnings: Real but elusive, and politically significant. Water Environ. J., 14, 7–14, doi:10.1111/j.1747-6593.2000.tb00219.x.
Pennsylvania Emergency Management Association, 2011: PEMA directive 2011-02: Emergency management certification and training requirements. Accessed 7 November 2011. [Available online at www.pema.pa.gov/Pages/Directives.aspx#.VVoslflVhBc.]
Perry, R. W., and J. M. Nigg, 1985: Emergency management for communicating hazard information. Public Adm. Rev., 45, 72–77. [Available online at www.jstor.org/stable/3135000.]
Pielke, R. A., 1999: Who decides? Forecasts and responsibilities in the 1997 Red River flood. Appl. Behav. Sci. Rev., 7, 83–101, doi:10.1016/S1068-8595(00)80012-4.
Rayner, S., D. Lach, and H. Ingram, 2005: Weather forecasts are for wimps: Why water resource managers do not use climate forecasts. Climatic Change, 69, 197–227, doi:10.1007/s10584-005-3148-z.
Swanson, H. D., 2000: The delicate art of practicing municipal law under conditions of hell and high water. N. D. Law Rev., 76, 487–510.
Verkade, J. S., and M. G. F. Werner, 2011: Estimating the benefits of single value and probability forecasting for flood warning. Hydrol. Earth Syst. Sci., 15, 3751–3765, doi:10.5194/hess-15-3751-2011.
Weaver, J., L. C. Harkabus, J. Braun, S. Miller, R. Cox, J. Griffith, and R. J. Mazur, 2014: An overview of a demographic study of United States emergency managers. Bull. Amer. Meteor. Soc., 95, 199–203, doi:10.1175/BAMS-D-12-00183.1.
Welles, E., S. Sorooshian, G. Carter, and B. Olsen, 2007: Hydrologic verification: A call for action and collaboration. Bull. Amer. Meteor. Soc., 88, 503–511, doi:10.1175/BAMS-88-4-503.
In this paper, the term short-term river forecasts refers to river forecasts of the NWS Advanced Hydrological Prediction Service, such as the example shown in Fig. 1. These forecasts are not to be confused with flash flood forecasts or flash flood guidance products.
The job title for EMs differs between states. For example, it is “emergency management director” in Oklahoma and “emergency management coordinators” in Pennsylvania. In all cases, the EMs coordinate the emergency response of the jurisdiction they are responsible for.
The Pittsburgh region was chosen because the authors’ home institution is located there. Arkansas and Oklahoma were chosen because the river forecast center there has been archiving the forecasts for more than a decade. Additionally, the contact information for all local (not just county) emergency managers in Oklahoma was available online.
It seems to be the perception of the interviewed EMs that the increasing number of full-time emergency managers is also a result of the influx of federal money since the terrorist attacks in 2001 and the advent of the Department of Homeland Security [e.g., federal preparedness (nondisaster) grants].
In 1990, the IAEM concluded among other things that 39 U.S. states had “no state mandated minimum qualification requirements for local emergency management coordinators” (IAEM 1990, p. 7). While this is not representative anymore, it does shed a light on the circumstances under which some of today’s traditionally long-serving EMs have been appointed to their jobs.
EMI courses in the 2014 course catalog that have the words National Weather Service, forecast, or weather in their course description or that focus on flood response (in italics) (FEMA 2014):
E0102 Science of Disaster;
IS-0247.a Integrated Public Alert and Warning System;
IS-0271.a Anticipating Hazardous Weather and Community Risk, 2nd edition;
IS-323 Earthquake Mitigation Basics;
IS-0324.a Community Hurricane Preparedness;
G0270.3 Expedient Flood Training;
G0271 Hazardous Weather and Flood Preparedness;
G0272/L0098 Warning Coordination;
G0361 Flood Fight Operations;
G0363/L3011 Hurricane Readiness for Coastal Communities;
G0365 WEM: Partnerships for Creating and Maintaining Spotter Groups;
L0320/L0324 Hurricane Preparedness for Decision Makers; and
V0007 Virtual Tabletop Exercise: Flood.
The only EMI course in the 2014 course catalog that is jointly taught by Emergency Management and NWS staff (FEMA 2014) is G0365 WEM: Partnerships for Creating and Maintaining Spotter Groups.
Probabilistic forecasts have uncertainty associated with them, just as deterministic forecasts do. For example, like deterministic forecasts, probabilistic forecasts have more uncertainty in the tails due to data sparseness.
Comparable to findings for the general public (Morss et al. 2010, 2008).
According to Nicholson (2006), the liability issues tend to be neglected in emergency management. Regulations differ between jurisdictions and type of employment (professional or volunteer). The legal background gets little attention in the education of EMs (Nicholson 2006), contributing to their risk-averse behavior (i.e., assuming the forecasts are correct at all times) in decision making.
