Abstract

Recommendations by the National Research Council (NRC), the National Institute of Standards and Technology (NIST), and Weather-Ready Nation workshop participants have encouraged the National Oceanic and Atmospheric Administration (NOAA) and the broader weather enterprise to explore and expand the use of probabilistic information to convey weather forecast uncertainty. Forecasting a Continuum of Environmental Threats (FACETs) is a concept being explored by NOAA to address those recommendations and also potentially shift the National Weather Service (NWS) from (primarily) teletype-era, deterministic watch–warning products to high-resolution, probabilistic hazard information (PHI) spanning periods from days (and longer) to within minutes of high-impact weather and water events. FACETs simultaneously i) considers a reinvention of the NWS hazard forecasting and communication paradigm so as to deliver multiscale, user-specific probabilistic guidance from numerical weather prediction ensembles and ii) provides a comprehensive framework to organize the physical, social, and behavioral sciences, the technology, and the practices needed to achieve that reinvention. The first applications of FACETs have focused on thunderstorm phenomena, but the FACETs concept is envisioned to extend to the attributes of any environmental hazards that can be described probabilistically (e.g., winter, tropical, and aviation weather). This paper introduces the FACETs vision, the motivation for its creation, the research and development under way to explore that vision, its relevance to operational forecasting and society, and possible strategies for implementation.

FACETs is proposed as an enhancement to and reinvention of the deterministic, teletype-era watch–warning paradigm of the United States toward one based on high-resolution, probabilistic hazard information.

In the mid-2000s, the National Weather Service (NWS) of the National Oceanic and Atmospheric Administration (NOAA) commissioned the National Research Council (NRC) to recommend ways in which the NWS could “more effectively estimate and communicate uncertainty in weather and climate forecasts” (NRC 2006) so as to improve public safety, property protection, and economic viability. Nine overarching recommendations were made for NOAA’s NWS, the Office of Oceanic and Atmospheric Research (OAR), and the entire weather enterprise. Citing the inherent uncertainty in weather and climate forecasts (NRC 2003), the NRC recommended enterprise-wide pursuits to improve communication of risk and uncertainty, including the use of new probabilistic products and services (recommendation 5; NRC 2006).

Six years later, the National Academy of Sciences released a two-part assessment of the NWS Modernization and Associated Restructuring (MAR; NRC 2012a) and “advice for the NWS on how best to plan, deploy, and oversee future improvements based on the lessons learned from the MAR” (NRC 2012b). To position NWS core capabilities for twenty-first-century technology and societal needs, recommendation I.d of the latter report challenged the agency to lead the meteorological community in effectively communicating forecast uncertainty by expanding the use of probabilistic forecasts derived from statistical postprocessing and numerical weather prediction (NWP) ensembles. This recommendation included a requirement to develop communications formats through “careful design using cognitive research.”

While the 2012 NRC reports were being developed, Weather-Ready Nation (WRN) workshops were held in Norman, Oklahoma, and later in Birmingham, Alabama. These workshops were held in response to concerns within the weather community (i.e., NOAA, emergency managers, academia, commercial weather enterprises, etc.) over the high number of fatalities and injuries and the extensive property loss from the 26–28 April 2011 Southeast U.S. tornado outbreak (NWS 2011a) and the Joplin, Missouri tornado of 22 May 2011 (NWS 2011b), despite advances in meteorological accuracy, science, and technology (NRC 1999). Out of this workshop arose a strong community commitment to increase the involvement of social, behavioral, and economic (SBE) science research in support of existing and emerging hazardous weather forecasting practices (Lindell and Brooks 2013). Twelve specific research recommendations were made by the participants (Lindell and Brooks 2012), including one to evaluate the construction of warning polygons using probabilistic contours (project C), another to better understand populations’ response to communicated uncertainty and risk (project E), and a third to apply social sciences in laboratory settings to gauge the effectiveness of different warning message elements (project F).

Although focused specifically on tornado warnings, the National Institute of Standards and Technology (NIST) report on the Joplin tornado (NIST 2013) recommended the evaluation of probabilistic warnings. In recommendation 16, “NIST recommends that technology be developed to provide tornado threat information to emergency managers, policy officials, and the media on a spatially resolved real-time basis to supplement the currently deployed official binary warn/no warn system.” Further elaborating, NIST recommended the NWS consider “providing frequently-updated, gridded, probabilistic hazard information, which could be merged with other GIS [geographical information system] information to provide better hazard information and reduce false-alarm rates.”

NOAA’s National Severe Storms Laboratory (NSSL) was assessing its own warning research and development programs during this period. Subjective assessments were conducted on challenges in the existing hazard forecasting programs (see the “Current hazard forecasting and communication challenges” sidebar) and societal, technological, and scientific trajectories under way in the nation’s quest for natural hazards resiliency (see the “Enabling trajectories” sidebar). These internal assessments, along with the WRN, NRC, and NIST recommendations [especially NRC (2012b) and NIST (2013)], led to exploration of a new, probabilities-based, SBE-sciences-informed paradigm for hazardous weather forecasting. This paradigm became known as Forecasting a Continuum of Environmental Threats (FACETs; Rothfusz et al. 2015b). Owing to its origin within NSSL, FACETs concepts began with a focus on severe convective weather, and our discussions herein have retained that essence. Given the aforementioned, multihazard recommendations of the community, however, we contend the described concepts of FACETs are extensible to any environmental phenomenon to which probabilities can be ascribed. Examples of such applications in nonconvective weather hazards are presented in the “Enabling trajectories” sidebar.

CURRENT HAZARD FORECASTING AND COMMUNICATION CHALLENGES

Mileti and Sorensen (1990), Brotzge and Donner (2013, and references therein), and Lindell et al. (2016) document challenges in the existing hazard forecasting system, including prediction, detection, forecaster decision-making, dissemination, and response. Below, we describe these and other challenges used to guide the development of the FACETs paradigm.

Challenge 1: Information voids and discontinuous information. The current approach to NWS watches and warnings are represented in Fig. SB1. In this time–space continuum, an event is the meteorological phenomenon for which NWS products are issued to protect life and property. In the time prior to such events, NWS national centers and local offices issue outlooks for synoptic- and subsynoptic-scale areas to identify the possibility of a particular weather phenomenon occurring. As the event time nears, watches are issued, followed by warnings as the event unfolds.

Fig. SB1.

Schematic of the time–space continuum for forecasting high-impact weather. (Adapted from H. Lazrus, NCAR.)

Fig. SB1.

Schematic of the time–space continuum for forecasting high-impact weather. (Adapted from H. Lazrus, NCAR.)

Galluppi et al. (2015) noted challenges with this product-centric, deterministic approach in the delivery of decision-supporting information. For example, the time between issuances of outlooks, watches, and warnings yields information voids during which the product users may need additional information. Even if the information is “nothing has changed,” it can still be valuable. To fill the voids and enhance services, NWS forecasters often communicate (un)certainty via graphical, web-based images (GraphiCasts), online chat forums (NWSChat), webinars, or simple phone calls. These methods augment official products and greatly improve the flow of decision-supporting information, yet only text-based products are part of the official NWS product stream, and end users must acquire supplemental, desired information from disparate, nonintegrated sources and media. FACETs research and development is exploring the dissemination (and value) of frequently updated, high-resolution PHI to fill the aforementioned data voids.

Challenge 2: Presumed needs and tolerances. End users have a wide variety of needs and risk tolerances, and there is no single hazard forecasting threshold sufficient to address them all (Morss et al. 2010; Senkbeil et al. 2013). Therefore, a forecaster issuing a deterministic product at what he or she deems the “right” moment may presume foreknowledge of the recipients’ needs and risk tolerances but is also responding to a complex mix of governing policies, forecaster–partner–recipient relationships, software designs, etc., all of which carry an inherent presumption of end-user needs. The variation in those needs is shown by Hoekstra et al. (2011), who found tornado warning recipients’ mean preferred lead time was 34 min, but that figure depended on region, age, gender, and a host of situationally relevant issues that vary across all demographic classes.

NWS forecasters generally limit the duration of deterministic watch and warning products and then follow up with new products, even if the forecaster thinks the event has a higher-than-average likelihood of being prolonged in the first place. Ending deterministic watches or warnings at a quasi-arbitrary time may not be of the greatest value to end users. Since the deterministic, product-centric, hazard forecasting system cannot fully address the disparate needs of end users, a forecaster’s issuance of any product from that system is inherently (and necessarily) based on a perceived, generalized understanding of end-user needs and thresholds. That said, changing that system to one with layers of continuous information also presumes a similar usefulness and usability. FACETs research is under way to determine if this paradigm improves upon the current system (Karstens et al. 2016; LaDue et al. 2016; Miran et al. 2017).

Challenge 3: High-resolution information through low-resolution media. The national warning paradigm changed in 2007 when the NWS moved from county-based to storm-based (polygon driven) warnings (Ferree et al. 2007; Jacks and Ferree 2007). This allowed NWS forecasters to provide greater definition as to the location of storm-scale threats within a county. This higher-resolution information, however, still relied on less-specific, lower-resolution broadcast media such as NOAA Weather Radio and county-wide tornado sirens. High-resolution information communicated through low-resolution broadcast/warning media loses the advantage of the new techniques (Ferree et al. 2007; Sutter and Erickson 2010; Nagele and Trainor 2012). Nagele and Trainor (2012) also note end users more readily identify with geopolitical borders than meteorologically driven features, so descriptions of storm-based polygons (e.g., via broadcast meteorologists) often revert to standard geopolitical references, thereby reducing the intended effectiveness of polygons. Fortunately, private-sector providers have found ways to take advantage of (and provide) the higher-resolution data through smartphone and computer applications (Coleman et al. 2011; Casteel and Downing 2013). Even with these technological advancements, work must be done to assure PHI can comport with important legacy broadcast systems.

Challenge 4: Oversized and multiphenomena warning areas. Thunderstorm wind gusts ≥25.7 m s–1 (50 kt, where 1 kt = 0.51 m s–1) and hailstones ≥2.4 cm (1 in.) verify NWS severe thunderstorm warnings. These same events are considered “covered” in tornado warnings (Polger et al. 1994), even though the possibility or existence of a tornado is the focus of those warnings. As a result, NWS forecasters typically draw tornado warning polygons to encompass wind and hail events if/when they coexist with a tornado threat. For a typical supercell, the entire storm (and projected path) is usually included in the tornado warning polygon, even though the tornadic threat area is much smaller than the polygon. Figure SB2 shows the 5-yr average of total (polygon) area warned for tornadoes was 2,152,556 mi2 (5,575,094 km2), compared to only 5,066 mi2 (13,121 km2) of actual tornado damage (0.2% of the total warned area). It is conceivable such a dramatic differential may be a contributing factor to public complacency (Wang and Kapucu 2008) and might be improved through refined, phenomenon-specific PHI communication.

Fig. SB2.

Normalized representation of the aggregated annual tornado warning area (red) and actual tornado impact area (yellow) averaged over 2008–12. (Courtesy P. Marsh and G. Carbin.)

Fig. SB2.

Normalized representation of the aggregated annual tornado warning area (red) and actual tornado impact area (yellow) averaged over 2008–12. (Courtesy P. Marsh and G. Carbin.)

Figure SB2 is presented to demonstrate the scale differences between warnings and actual impacts. We acknowledge the figure employs several assumptions, namely 1) the area of tornado damage was calculated from maximum path width, so the actual impacted area is likely much smaller; 2) polygons (warnings) are issued sequentially in time during an event that is not factored into an aggregated spatial image; and 3) the calculation assumes society only perceives “impact” if they are in the path of the tornado. Howe et al. (2014), for example, found survey respondents’ perceptions of having been impacted by tornadoes varied with tornado strength, so the perceived impact area can be larger than shown. Even considering these assumptions, Fig. SB2 suggests polygon-driven warnings are blunt instruments for communicating dynamic, small-scale, multiphenomena threats. Smaller warning areas have been shown to have some measure of economic and societal benefits (Sutter and Erickson 2010; Nagele and Trainor 2012).

Challenge 5: Frame of reference. The current hazard-forecasting paradigm was designed by meteorologists. It is not surprising then that the paradigm is largely influenced by what meteorologists see on their computer workstation “frame of reference.” Their view of the “real world” is primarily through the spatial and temporal lenses of radar, satellite, model output, etc., while being informed by collective interpersonal relationships with a variety of end users and partners. The relevant frame of reference of an individual end user, however, is their personal (Lagrangian) space and their primary concern is likely, “How will this phenomenon impact me where I am (or where I am going)?” (Galluppi et al. 2015). Brotzge and Donner (2013) describe this as the personalization of warnings in which “warnings must evoke a sense of specific and immediate risk.” Our intention with FACETs is to improve the personalization of risk through information delivered within the context and reference frame of the individual end user (see the “Trajectory 5: Enabling output and SBE science involvement” section in the “Enabling trajectories” sidebar).

ENABLING TRAJECTORIES

The following scientific, societal, and technological trajectories were considered enablers and drivers in the development of the FACETs paradigm.

Trajectory 1: Probabilistic hazard information. Several members of the scientific community have advocated a shift to conveying uncertainty through probabilistic messaging (NRC 2006; AMS 2008; Hirschberg et al. 2011; NRC 2012b; National Academy of Public Administration 2013; NIST 2013). Karstens et al. (2015) proposed probabilistic hazard information as a manner of communicating threat probabilities. Animating PHI yields threats in motion fields (Stumpf et al. 2015) in which storm-based warning information is rapidly updated to address the temporal gaps between legacy warnings. Dance et al. (2010) developed similar concepts in their Thunderstorm Environment Strike Probability Algorithm (THESPA).

Likewise, local NWS WFOs have explored new methods of communicating hazard information in a gridded format through experimental “threat petals” (Frederick and Amburn 2015) and “plumes” (Mitchell 2014). The SPC is extensively exploring the use of grid-based probabilistic watch and outlook information (Kay and Brooks 2000; Sobash et al. 2011), and the broadcast media have introduced probability-implied concepts such as the tornado condition (TOR:CON) index of The Weather Channel (see weather.com/tv/shows/amhq/news/tornado-torcon-index). NOAA’s WPC and local forecast offices in the northeastern United States have been testing probabilistic winter weather forecasts (Novak et al. 2014; Waldstreicher et al. 2017). NOAA’s National Hurricane Center has been using strike probability grids for many years (Sheets 1985) and introduced probability-derived, local surge inundation maps in 2014 (Morrow et al. 2015). Morss et al. (2008) and Joslyn and LeClerc (2013) determined end users showed preferences of probabilistic over deterministic forecasts, although challenges exist in delivering such information (Novak et al. 2008; Donner et al. 2012), perhaps even more so in rapidly changing weather. The similarity between these proposed (or active) probabilistic hazard-forecasting methodologies suggests there is considerable interest and value in the issuance of PHI.

Trajectory 2: Enabling guidance. New NWP approaches such as the High Resolution Rapid Refresh (Benjamin et al. 2016) and Warn-on-Forecast signal future warning operations guided by short-range, small-scale NWP ensemble systems as well as hazard detection via remote sensing tools (e.g., radar; Stensrud et al. 2009). The vision of WoF, specifically, is to supply forecasters with 1-h guidance of explicit storm evolution, as derived from the agreement of storm-scale ensemble members. This guidance, owing to its ensemble modeling origins, is being supplied as probabilistic information (Yussouf et al. 2015). This will be a fundamentally new form of guidance in NWS severe weather operations and provides an opportunity to augment deterministic, text-based warnings with more sophisticated information.

Trajectory 3: Enabling domains. Science and society are turning to grid-based, digital information in a variety of formats. The NWS has made important progress in this regard, establishing the National Digital Forecast Database (NDFD; Glahn and Ruth 2003), which is accessible and open to external users of NWS forecast information. The NDFD is populated by NWS meteorologists creating their routine forecast data (surface temperatures, winds, probability of precipitation, etc.) on the AWIPS GFE. While hazardous weather watches, advisories, and some long-fused warnings are created using the GFE, the approach has not translated downscale to grid-based, short-fused warning information primarily due to technological, scientific, and service delivery challenges.

Trajectory 4: Enabling tools. Several NWS WFOs have experimented with using GFEs to create small-scale, short-term forecasts for public and aviation purposes (Palmer et al. 2013; Waldstreicher et al. 2012). While GFE tools were not specifically designed to create forecasts at such small temporal and spatial scales, the efforts were undertaken to satisfy end-user demands for high-resolution information. Next-generation software for AWIPS is being developed by NOAA’s Global Systems Division based on research and development work done by Karstens et al. (2015). This GSD warning tool, known has Hazard Services, is being developed so forecasters can create and deliver high-resolution PHI (Argyle et al. 2017).

Trajectory 5: Enabling output and SBE science involvement. The NWS (and others) have undertaken projects to improve societal response to watches and warnings by improving the messaging (or output) by which end users make decisions (League et al. 2010; Bunting 2015; Casteel 2016). The NWS Hazard Simplification project is a recent example (Jacks et al. 2013), while new experimental probabilistic guidance products have been presented to the public by national centers [e.g., SPC, WPC, and the National Hurricane Center (NHC)] as early as the 2000s. SBE sciences are being incorporated into the processes of message design and response evaluation (Ash et al. 2014). The emergence of SBE sciences as a vital component in the entire warning system, not just message design, is another important trajectory under way within the meteorological community (e.g., Gladwin et al. 2009; Lazo 2010; Trainor et al. 2015; Lindell and Brooks 2012). Efforts like WAS*IS (Demuth et al. 2007), SSWIM (Gruntfest and Lazrus 2009), and the 2015 University of Oklahoma/NOAA Living with Extreme Weather Workshop (Droegemeier et al. 2016) are working to foster these multidisciplinary interactions.

The primordial soup—And an opportunity for coalescence. As shown in these trajectories, myriad activities have been and are being conducted to improve societal response to hazardous weather information. Each has its own pulse, energy, direction, and purpose and collectively comprise a veritable primordial soup of activities often disconnected from a broader, organizing framework. We offer FACETs as that framework to minimize duplication of effort, coordinate activities, fill gaps in physical and SBE scientific evaluation, and avoid failed (or nominally successful) operational application. Our vision for FACETs is to be an effective, modern, unifying, scientifically robust, holistic, and enabling paradigm responsive to the aforementioned trajectories and WRN-identified needs.

A common theme within the aforementioned studies is an acknowledged potential for societal benefit through the forecasting and communication of probabilistic hazardous weather information. Deliberate and rigorous scientific research and development are under way with ongoing FACETs work to determine with reasonable certainty whether the cited recommendations will be of value to the nation. As described below, this work depends on a combination of carefully controlled experiments, technological developments, and naturalistic (i.e., real world) investigations.

BACKGROUND.

The first official severe convective storm watches and warnings were issued by the U.S. federal government through the U.S. Weather Bureau (USWB) in 1965, although some USWB offices were issuing them in an experimental fashion as early as 1956 (Coleman et al. 2011). Additionally, the USWB’s Severe Local Storms (SELS) unit issued tornado forecasts, the predecessor to today’s tornado watches, starting in 1952 (Edwards and Ostby 2009). Subsequent improvements in severe convective watches and warnings of the USWB, later renamed the NWS, have generally centered on advancing hydrometeorological science, NWP, and remote sensing technologies. These efforts improved event predictions, hazard detection, and warning lead times (e.g., Polger et al. 1994; Golden and Adams 2000; Simmons and Sutter 2005).

Advances in the communication of weather hazards generally followed advances in modernized telecommunication and broadcast technologies, including the introduction of NOAA Weather Radio dissemination in the 1960s, standardized bullet-format products in the 1990s (NWS 2012), and storm-based, or polygon, warnings in 2007 (Ferree et al. 2007). The quality, reach, and impact of broadcast media, the Internet, and social media transmissions of these products improved correspondingly (Coleman et al. 2011).

Advances in our scientific knowledge, observational capabilities, communication infrastructure, and warning methodologies have improved NWS services to the public even though official hazardous weather products (defined herein as watch–warning messages prepared and disseminated by NWS forecasters) remain generally text based and deterministic. These products are issued when forecasters determine a threshold of severity has been or will be met or surpassed. NWS forecasters regularly and effectively augment these products with other decision-supporting information based on meteorological insights, interpersonal relationships, and societal needs, but official NWS watch and warning products remain largely text based (see the “Challenge 1: Information voids and discontinuous information” section in the “Current hazard forecasting and communication challenges” sidebar).

Benefits and limitations resulting from this current service paradigm have been described in NWS service assessments of high-impact weather events (e.g., NWS 1994, 1999, 2009, 2011a,b, 2014). Quantitative measures of that service have focused on the economic value achieved by technological advances (Simmons and Sutter 2008; NRC 2012a) and less so on the value of any underlying, deterministic (or probabilistic) forecast methodologies. A goal of FACETs research is to determine and compare the value and merits of new and legacy forecast methodologies.

PROBABILISTIC INFORMATION AND SOCIETAL RESPONSE.

Recent research highlights opportunities for including probabilistic information to enhance the deterministic watch–warning system. Research on interpretations of deterministic forecasts find that many people already infer uncertainty into deterministic forecasts for field properties such as temperature and would prefer to have ranges of uncertainty provided by forecasters (Morss et al. 2008). Correspondingly, there are indications core NWS partners make decisions that differ from thresholds set for binary warning decisions at NWS forecast offices; for example, many emergency managers sound sirens even without tornado warnings, and occasionally they do not sound sirens when there are tornado warnings (League et al. 2010). This behavior implies i) differing risk tolerances and adaptabilities across communities, ii) end users are making inferences about thresholds that concern them, and iii) more detailed estimates of hazard probability may help these specialized users refine their responses. Research in controlled experimental settings further demonstrates some of the potential value in communicating probabilistic information to wide audiences, as participants who are provided with probabilistic information in certain formats can become better calibrated to the threats they face than those who are presented with deterministic information alone (Joslyn et al. 2007; LeClerc and Joslyn 2012; Joslyn and LeClerc 2013).

This research is at a relatively early stage, and the full implementation of continuously evolving probabilistic information may pose new challenges that are not yet well understood. Graphical, grid-based, probabilistic hazard information (PHI), for example, is likely to experience some of the same challenges as documented in interpreting the hurricane cone of uncertainty (Broad et al. 2007), even if the probabilities belong to different reference classes. In fact, these challenges may be amplified in a continuously evolving situation; mesoscale PHI plumes (graphical depictions of specific thunderstorm phenomenon probabilities; see Fig. 1) may not move along smoothly, for example, fluctuating in shape, orientation, and magnitude, which would pose challenges for interpretation of risk within a place-based, Eulerian reference frame (not just following the storm). Research on the provision of uncertainty information in other domains such as environmental health reveals that while certain probabilistic formats can reduce cognitive bias and improve calibration, this relies on the ability of the person to process probabilistic information in the first place (e.g., possess high numeracy; NRC 2013). Advances in probabilistic tools that do not consider the needs and abilities of multiple populations, especially the most vulnerable (often low-numeracy populations), will likely fall short of their objective.

Fig. 1.

Example representation of a legacy warning (red area) and possible new alert products (yellow) derived from PHI grids. The meaning of the probability values is undefined for this example and needs to be clearly defined through FACETs research.

Fig. 1.

Example representation of a legacy warning (red area) and possible new alert products (yellow) derived from PHI grids. The meaning of the probability values is undefined for this example and needs to be clearly defined through FACETs research.

Another concern related to the weather response infrastructure described earlier is the extent to which this information will not lend itself to straightforward applications within legacy communication systems and/or cause disruption and confusion within systems that can broadcast it. For example, updates to forecasts ripple through social media in complicated ways (Morss et al. 2017). These challenges require substantial research efforts that include a suite of basic/experimental and naturalistic methodologies alongside concerted enterprise-wide involvement in the development process.

FACETs.

FACETs is simultaneously a framework and a new hazard-forecasting paradigm. It is i) a framework for organizing and testing improvements in the hazard forecasting system that, if implemented, may result in ii) a new paradigm in which intermittent, deterministic hazardous weather products are augmented by a continuous flow of high-resolution probabilistic information. Although initially developed for severe convective weather forecasting (Kuhlman et al. 2008; Karstens et al. 2015), the notion of PHI may be extensible to a wide range of environmental phenomena (see the “Facet 1: Method and manner” section below and “Trajectory 1: Probabilistic hazard information” section in the “Enabling trajectories” sidebar).

Doswell et al. (1999) defined the components of the hazardous weather warning process as forecast, detection, warning decision, dissemination, and public response. Our framework offers an expanded version of this process to include seven, interrelated components:

  1. method and manner—the foundational design of the hazard forecasting practice (e.g., deterministic, text-based information associated with storm-based polygons);

  2. observations and guidance—the tools and data by which hazard diagnoses and predictions are achieved (e.g., radar, satellites, meteorological observations, numerical weather prediction, and statistical guidance);

  3. the forecaster—the knowledge, skills, abilities, relationships, and cultures of forecasters needed to make effective forecast decisions;

  4. tools—the software, hardware, and systems used to create and disseminate the hazard forecast information;

  5. output—the format, content, equipment, and media by which the hazard forecast information is communicated;

  6. response—all aspects of the recipient’s response (or nonresponse), including all factors leading up to and following the receipt of the message [e.g., understanding, belief, confirmation, personalization of risk, action (Brotzge and Donner 2013), education, preparedness, situational awareness, and recovery]; and

  7. verification—quantitative and qualitative measures taken to validate the scientific integrity and effectiveness of the hazard forecasting system and to inform improvements in that system.

The order of these components matches the typical forecast process (see Fig. 2). For example, once the hazard forecasting methods have been established, which is done by the disseminating organization well before any weather event is forecast, observational and guidance data are collected for the forecaster to evaluate and make a decision, after which tools are used to create output and elicit a desired response. Finally, verification of forecast accuracy and societal benefit is conducted.

Fig. 2.

Schematic of the components of the forecasting process, which also serves as the organizing framework for the FACETs paradigm.

Fig. 2.

Schematic of the components of the forecasting process, which also serves as the organizing framework for the FACETs paradigm.

This taxonomy was created to provide an organizing framework for the development and evaluation of PHI in the hazard forecasting system. While the existing deterministic system could be tweaked in a piecemeal fashion without such a framework, implementing a fundamentally different method and manner (“Facet 1: Method and manner” section), as proposed in FACETs, would inevitably affect the entire system. In other words, evolving the deterministic, yes–no warning practice currently employed toward use of high-resolution (e.g., ≤5-km grid spacing and ≤2-min update frequencies) PHI would impact downstream facets (Karstens et al. 2015). The authors, therefore, found value in developing this highly interdependent, seven-component structure to organize, guide, and focus the comprehensive work necessary to properly evaluate PHI concepts for possible operational implementation. The interconnectedness and overlap of our framework facets are acknowledged and emphasized, with awareness that specific forecast system aspects (e.g., data formats) could reside in multiple facets (e.g., methods, tools, or output). For the purpose of discussion, however, the seven-facet structure has proven helpful.

A second foundational aspect of FACETs is the integration of appropriate SBE science disciplines (e.g., applied anthropology, economics, human factors, sociology, communication, human geography, political science, linguistics, and history) into the stages (facets) of the hazard forecasting process (Fig. 2) as it is being developed or improved. Because hazardous weather forecasting is a process in which humans are inexorably involved and impacted, early consideration of SBE science in a new paradigm like FACETs is just as important as physical science considerations. This is in keeping with a WRN recommendation to include interdisciplinary program evaluations in all stages of emerging forecast–warning systems development (Lindell and Brooks 2013). Applying SBE sciences to real-world, physical science problems is not new (e.g., Mileti and Sorensen 1990), but doing so in a robust and truly integrated fashion is [e.g., Weather and Society Integrated Studies (WAS*IS; Demuth et al. 2007; Kuhlman et al. 2009), Social Science Woven into Meteorology (SSWIM; Gruntfest and Lazrus 2009), and as described in Schumacher et al. (2010)]. Therefore, just how, when, and where such integrations can and should (or should not) occur must be guided by continued, thoughtful scientific discovery between SBE and physical scientists. Early FACETs research is being guided in this way through SBE integration in NOAA’s Hazardous Weather Testbed (HWT; e.g., Ling et al. 2015; Karstens et al. 2015; Miran et al. 2017) and collaborative interaction with SBE research colleagues (e.g., Droegemeier et al. 2016).

As its name implies, Forecasting a Continuum of Environmental Threats is intended to apply to the forecasting of all environmental threats, regardless of impact, cause, and/or location. The continuum of FACETs refers to those environmental threats occurring across all temporal and spatial scales (e.g., seasonal to thunderstorms), that are impactful to society. Winter weather, fire weather, flooding, severe convective storms, and other naturally occurring environmental hazards can be included in the continuum and, while each has different spatial–temporal scales, climatologically favored locations, and impacts, all are intended to be addressed in FACETs. For some weather phenomena and spatiotemporal scales, probabilistic information is already (or is a growing presence) in forecasts; for example, probabilistic renderings of snowfall potential, rainfall totals, and day-of forecasts for severe weather hazards are provided by NWS forecasters at national centers and reflected in downstream communications by partners throughout the weather enterprise. However, making these practices routine within the NWS remains a key challenge for all hazards (Jacks et al. 2013), and questions remain about how these forecasts and their communication should change with the variety of emerging probabilistic guidance available. Although focused on weather-related phenomena for our purposes herein, we envision FACETs concepts might even apply to other natural or anthropogenic phenomena to which probabilities can be assigned (e.g., earthquakes, tsunamis, volcanoes, solar flares, diseases, and pollution impacts).

FACET 1: METHOD AND MANNER.

Sanders (1963) described probabilities as “the internal language of forecasters.” Murphy (1991) articulated the advantages of probabilistic forecasts and verification over those of categorical (deterministic) forecasts. Others followed to express inherent benefits and value in probabilistic forecasts (e.g., Hamill 1997; Doswell 2004; NRC 2012b). FACETs is built upon these works and proposes PHI as a method and manner for communicating threats in high-impact weather events.

For the purposes of FACETs, we define PHI as the probability of a weather-related event occurring within a spatial and temporal range. In that regard, PHI represents probabilities related to the occurrence of a specific weather phenomenon [e.g., 1-in. (2.54 cm)-diameter hail, a tornado, lightning, snow or rain accumulation of a particular amount, freezing rain, or a specific temperature] or more complex information such as the probability of a phenomenon’s arrival, onset, or ending time or its magnitude. The event of interest and the context within which it occurs, then, determine what PHI is describing (i.e., the reference class of the probability). By this definition, there could be countless and overlapping possibilities of PHI for a given weather situation. The most relevant and impactful PHIs must be chosen for each weather scenario to ensure forecasters’ and end users’ needs are met. Early work in the HWT, for example, showed forecasters had difficulty creating PHI without first-guess data derived from model output or statistical analyses (Stumpf et al. 2008). Ongoing PHI research (e.g., Karstens et al. 2015) is exploring if forecasts initialized in this way can be monitored and updated utilizing standard observation and guidance information (see “Facet 2: Observations and guidance” section) and adjusted using grid-manipulation tools (see “Facet 4: Tools” section), a process known as forecaster over the loop (Roberts et al. 2012).

The FACETs paradigm does not preclude other means of communicating uncertainty through qualitative and/or legacy (polygon and text based) products (e.g., watches and warnings) if SBE research findings indicate their societal benefit. A refinement of those legacy warnings may be possible, however, by having phenomenon-specific (e.g., hail or heavy snow) warnings automatically and more frequently extracted from PHI databased on predetermined threshold values (Fig. 1). This is a likely improvement over today’s practice in which one warning type (e.g., a tornado warning) addresses multiple phenomena (see the “Challenge 4: Oversized and multiphenomena warning areas” section in the “Current hazard forecasting and communication challenges” sidebar) but will require significant investigation of a range of methodologies. To that point, HWT experiments (LaDue et al. 2016) have explored delivering decision support information using PHI even while addressing new messaging practices of the NWS Hazard Simplification project (Jacks et al. 2013).

FACETs research will explore if forecasters can rely less on their own deterministic “issue–do not issue” decisions and, by continuously updating PHI, deliver better, nuanced threat information decision-makers need (NIST 2013; Karstens et al. 2015). Through PHI, sophisticated end users (e.g., hospitals, nursing homes, and large venue facilities) could set their own probability and/or lead-time thresholds for action based on their specific needs and risk tolerance (see the “Example of matching PHI with user needs” sidebar). Here, too, significant SBE research will need to occur to determine proper thresholds. Given the significant potential for new services and products afforded by these studies and PHI (see the “Facet 5: Useful output” section), the commercial weather enterprise could have tremendous opportunities for new and/or enhanced services derived from these data.

EXAMPLE OF MATCHING PHI WITH USER NEEDS

Nursing-home residents are vulnerable to negative health impacts and morbidity during evacuation, shelter-in-place activities, and significant disruption in daily activities (e.g., Brown et al. 2012; Rothman and Brown 2007). It is therefore infeasible for nursing homes to commence effective evacuation or shelter-in-place protocols only when a “go” message is received in the form of a deterministic warning. Thus, the current hazardous weather paradigm and its binary output poorly match the nuanced, situation-dependent evacuation or shelter-in-place practices of nursing homes described by the Florida Health Care Education and Development Foundation (2008). This is but one example wherein development of comparably nuanced, calibrated PHI would allow weather enterprise providers and their clients to establish thresholds and action criteria based on their specific needs. The same can be said for any organizations where hazardous weather risk tolerance is low and lead-time needs are high.

FACET 2: OBSERVATIONS AND GUIDANCE.

This facet represents the broad array of tools and technologies used by forecasters to make hazard-forecasting decisions. It includes remote sensing tools (e.g., radars and satellites), meteorological observations, public reports, NWP, statistical guidance, and even forecaster-to-forecaster interaction. Owing to its breadth, diversity, and underlying purpose of informing forecasters on the present and future states of the atmosphere, this facet is the core of any hazardous weather forecasting system and, thereby, has received the bulk of community research and development attention compared to other facets.

NWP ensembles are growing sources of guidance being developed to aid in weather forecasting (see the “Trajectory 2: Enabling guidance” section in the “Enabling trajectories” sidebar). Output from these models is becoming increasingly probabilistic as well and will provide opportunities for NWS forecasts to employ PHI in new ways, as recommended by the scientific community (see “Trajectory 1: Probabilistic hazard information” in the “Enabling trajectories” sidebar). FACETs was created, in part, as a framework to guide delivery of NWP ensemble-generated probabilistic output such as that from Warn-on-Forecast (WoF; Stensrud et al. 2009) models.

There are other burgeoning sources of PHI guidance in the FACETs research domain. Reanalyzed radar data, for example, Multiyear Reanalysis of Remotely Sensed Storms (MYRORSS; Cintineo et al. 2012) data, provide real-time, statistical projections of a thunderstorm’s longevity, intensity, and attendant phenomena (Ortega 2015; McGovern et al. 2016). ProbSevere model data (Cintineo et al. 2014) provide real-time, probabilistic projections of storm severity based on satellite and Multi-Radar Multi-Sensor (MRMS) system data (Zhang et al. 2016; Smith et al. 2016). MYRORSS and ProbSevere are on development timelines to serve as precursors to WoF-derived probabilistic guidance.

Guidance comes from forecaster-to-forecaster interaction, as well. While such interactions may work smoothly within an NWS office, interoffice coordination of watches and warnings may be complicated by geopolitical boundaries, office cultures, and local needs (Reeves et al. 2013). Grid consistency is a challenge for synoptic-scale forecasts and will be made even more challenging with ever-decreasing time and space scales. Forecast consistency is further complicated when national centers such as the Weather Prediction Center (WPC) do not operate on the same forecast grids as local NWS Weather Forecast Offices (WFOs). It is logical, then, to consider using single, shared grids of PHI to ensure forecast consistency across temporal and spatial domains, such as the approach being explored in the NWS National Blend of Models (NBM; Gilbert et al. 2015). NOAA’s Storm Prediction Center (SPC) and WPC have begun exploratory work in providing guidance to WFO forecasters via grid-based, probabilistic outlooks, watches, and discussions (Bunting 2015; Barthold et al. 2015). FACETs research is exploring the feasibility of having a national, grid-based PHI flow downscale to populate local WFO PHI grids.

FACET 3: THE FORECASTER.

Operational forecasters provide expertise needed by decision-makers in all weather scenarios. The forecasters’ skills, knowledge, interpersonal relationships, and passion are instrumental in the success of weather-related decisions made by end users. A goal of FACETs is to augment and enhance those abilities and traits through the use of PHI and applied SBE sciences. Although PHI guidance is being developed to assist the forecaster, total automation of the forecast process is not a priority in the FACETs paradigm. In fact, PHI guidance will likely never be perfectly calibrated, which keeps the skills of the forecaster at the forefront of the forecasting process. PHI guidance is intended to assist properly trained forecasters in more effectively communicating forecast certainty. A fighter pilot analogy could be made here. Despite the impressive technologies used in fighter jets, the pilot is still at the heart of the flying process (even with drones) and the supporting technologies are used to improve the effectiveness of the pilot. The same can be said for forecasters making increasingly nuanced forecasts aided by PHI-generation tools. To achieve this, there are critical social and behavioral science questions to be explored with regard to forecasters’ curation of PHI in a variety of meteorological (e.g., supercell vs nonsupercell), societal (e.g., end-user risk comprehension and the forecasters’ understanding of it), and geographic (e.g., urban vs rural) contexts. FACETs intends to keep the forecaster at the heart of the process, while exploring how PHI can make that forecaster most successful.

The FACETs paradigm, if implemented, will bring cultural changes to NWS operations. The HWT work of Kuhlman et al. (2008) and Karstens et al. (2015) suggests some forecasters may have difficulty moving from a deterministic, product-centric mode in which a definitive hazard forecasting decision is made by the forecaster to one in which such legacy products “fall out” of the forecaster-adjusted PHI. The need for considerable forecaster training on probabilistic threat information, uncertainty conveyance, use of new guidance resources, and data-creation tools, etc. was evident in these studies. Forecaster adaptation may go beyond the mechanics of PHI manipulation. From our operational experience, forecasters take great pride in being the one to make the (deterministic) watch and/or warning decisions and rightfully so, as such decisions can be lifesaving. It is unknown whether forecasters will respond more or less favorably to the over-the-loop act of adjusting PHI fields to generate deterministic products and/or whether the final outcomes will be impacted. Applying SBE sciences to such questions is a major goal of FACETs.

FACET 4: TOOLS.

This facet applies to the tools forecasters use to ingest, manipulate, update, and disseminate PHI. Presently, the Advanced Weather Information Processing System (AWIPS) Graphical Forecaster Editor (GFE; Hansen et al. 2003) is used by the NWS to create routine forecast grids of sensible weather (e.g., wind, temperatures, sky cover, and precipitation probabilities) and some hazardous weather grids for watches and nonconvective warnings. Although AWIPS was originally designed and built through thoughtful integration of physical and SBE sciences (e.g., Hoffman 1991; Doswell 1992; Lusk et al. 1999), it has limited capabilities for creating, displaying, and disseminating the PHI needed within the FACETs paradigm. NOAA’s NSSL, NWS, and Global Systems Division (GSD) are collaborating on AWIPS Hazard Services development with these PHI concepts in mind (Argyle et al. 2017). Sophisticated, science-based “recommenders” (first-guess indicators of hazard thresholds being exceeded) are being designed in Hazard Services to facilitate rapid decision-making and PHI creation. Human-factors scientists are also assisting in Hazard Services software development to analyze the design and effectiveness of forecast tools for creating and issuing PHI (Ling et al. 2015).

To maximize the applicability in the weather enterprise, FACETs research is exploring open-data formats, with grid-based PHI being one preferred format. Severe convective watches and warnings, for example, are not currently issued via grids or open data formats, but GSD is developing Hazard Services to support this. If designed and implemented properly, GIS technologies and other grid methodologies already in use (e.g., the U.S. National Grid administered by the Federal Geographic Data Committee [FGDC 2001]) could apply and exploit PHI.

FACET 5: USEFUL OUTPUT.

The U.S. weather enterprise, its customers, and other NWS stakeholders and end users have been accustomed to receiving and using deterministic products. Replacing them completely with PHI before a new paradigm is proven would be imprudent. A cautious evolution to communicating more specific (and probabilistic) information in watches and warnings is the preferred approach. Although PHI is at the heart of the FACETs paradigm, our research is also investigating whether other information (e.g., forecaster confidence, crowd-sourced observations, expected impact magnitudes, and range of impacts) might also add value to legacy hazard forecasts. SBE research conducted with emergency managers (LaDue et al. 2016) and the general public (Ash et al. 2014; Miran et al. 2017) is among the early steps to understanding the effectiveness of PHI-driven output in the FACETs paradigm. Although nascent and needing subsequent work, these studies have indicated the value of PHI within certain contexts while showing the need to retain some degree of determinism in FACETs-era forecasts. As such research continues, new insights will inevitably refine future applications of PHI output

An overarching goal of FACETs is to deliver a continuous, rapidly updated stream of PHI at high spatial resolutions from months to minutes prior to an event, filling gaps in the existing, deterministic system. We propose FACETs as a framework for delivering a continuum of PHI (which may include deterministic, legacy products). The power of FACETs, we contend, is in the ability of sophisticated recipients and value-adding enterprises to “mine” user-specific, actionable information from this high-resolution continuum of data so as to feed a wide variety of probabilistic and deterministic displays, formats, and applications for an equally wide variety of end users.

With the aid of insights from SBE science research on public comprehension and behavior, and the creativity of weather enterprise entrepreneurs, new PHI formats and content might include the following:

  • neighborhood-resolution meteograms of threat probabilities for various phenomena;

  • highly adaptive, personalized smartphone apps to describe threat trends (and possibly even nearby shelter locations);

  • specific impact possibilities as derived by cross-checking PHI with vulnerability catalogs (e.g., Scharfenberg et al. 2015);

  • representations of forecaster confidence (Karstens et al. 2015); and

  • confidence-driven threat categories for weather hazards.

This last example is being employed via Meteoalarm in Europe (Niedermoser et al. 2007), and the NWS is exploring similar approaches through its Hazard Simplification project (Jacks et al. 2013). FACETs research is under way to determine if/how such categorical information could be mined from PHI (LaDue et al. 2016).

FACET 6: EFFECTIVE RESPONSE.

Any progress made in the previous five facets would be for naught if the end-user response results in avoidable death, injury, or property loss. This facet is where the physical science of weather forecasting intersects humanity (society) and, as a result, is where the application of SBE sciences can have the greatest impact. The WRN meetings illuminated wide-ranging questions about an “effective and appropriate” end-user response at this intersection (Lindell and Brooks 2013).

FACETs research is addressing this by integrating multidisciplinary SBE expertise across all facets, with particular attention to this one (Ling et al. 2015; Karstens et al. 2016; LaDue et al. 2016). As understood by many in the SBE community (e.g., Mileti and Sorensen 1990; Lazo et al. 2009; Howe et al. 2014; Lindell et al. 2016; Galluppi et al. 2015; Morss et al. 2016), learning how people respond to weather risk is crucial to designing a system that will achieve the desired outcomes. Insights from research in this area will be used to inform choices made about the technologies designed and implemented in other facets, as well.

FACET 7: VERIFICATION.

The value and effectiveness of any forecast system are determined through robust and meaningful verification of its processes and outcomes. FACETs research is exploring the development of new performance metrics by comparing forecasts and observations on a common geographic reference system, unlike legacy product verification today. Brier scores, false-alarm duration, false-alarm area, site-specific lead time, site-specific end time, and other metrics are being investigated for their utility in providing greater insight into the effectiveness of hazard forecasts (Stumpf et al. 2015). This work will also include exploration of high-resolution proxies, such as radar-derived hail estimates, to provide an alternate assessment of event occurrences when observed ground truth does not exist (Snook et al. 2016).

Standard metrics of severe weather forecasting success such as those established by the Government Performance and Results Act (GPRA) focus on forecast skill and serve indirectly as a measure of societal value. With SBE at the core of FACETs, regular measurements of societal response and value are envisioned, as proposed by Lazo et al. (2009). Regular measurements of user response would be a new aspect in NWS verification practices, as such evaluations are typically reserved for formal (and infrequent) postevent service assessments. Appropriate measures and data-collection techniques would need to be devised through collaboration with SBE scientists, emergency management, and weather enterprise partners. Effective, scientifically valid, and low-workload methodologies to accomplish this are being explored in FACETs research. Studies of the public’s response to hazard communications (e.g., Morss et al. 2008; Ripberger et al. 2015; Morss et al. 2016) can also serve as potential baselines to evaluate response differences between recipients of deterministic information and PHI.

Delivery of calibrated probabilities is a key goal within the FACETs paradigm. Evaluations of probability verification statistics will aid in determining hazard forecast reliability, focus forecaster training, identify calibration needs, improve end-user cost–loss decision-making and, thereby, enhance forecast value for users with well-defined hazard probability thresholds and adaptive capacities necessary for harnessing the information (Murphy 1991, 1993). Such complex insights are much more difficult to garner from legacy statistics (e.g., probability of detection and false-alarm ratio) that reflect decision criteria based on forecasters’ general knowledge of the risk preferences of the diverse populations being served.

Owing to its nascency, the relative merits and weaknesses of the FACETs paradigm over legacy forecasting practices are not fully known, but insights are being gained through testbed and focus group evaluations (Karstens et al. 2015). In addition, side-by-side evaluations of standard GPRA metrics, the measures proposed above (e.g., end-user response, Brier scores, and false-alarm areas), and material societal costs (e.g., lives and property lost) must be conducted within both legacy- and FACETs-era contexts to understand the merits and weaknesses of each paradigm and whether FACETs concepts deliver the anticipated improvements. FACETs research is planned (and under way) to conduct such rigorous, bimodal comparisons.

IMPLEMENTATION OF FACETs CONCEPTS.

Hazardous forecasting and communication improvement activities are under way within NOAA (e.g., WoF, Hazard Simplification, Hazard Services tools, and experimental probabilistic forecast products) but lack an obvious, overarching vision for a future hazard-forecasting paradigm. In June 2014, representatives from NOAA operations, research, and leadership; the commercial weather industry; SBE sciences; and other weather community stakeholders participated in a workshop in Norman to create a Science and Strategic Implementation Plan (SSIP) as a guide to developing, exploring, and possibly implementing FACETs in severe convective weather forecast operations (Rothfusz et al. 2014a). The SSIP lays out a multiyear, multidisciplinary “project of projects” strategy to evaluate FACETs as a viable, modernized approach to forecasting and communication in high-impact environmental events. The plan identifies an initial 46 projects necessary to expand the current, deterministic paradigm to that of FACETs. The NWS is also exploring the application of PHI and the FACETs paradigm in other program areas (e.g., hydrologic, winter, tropical, aviation, fire weather, marine, and public).

The NOAA HWT, Hydrometeorological Testbed, and Operations Proving Ground will foster insights into the challenges, benefits, possibilities, and feasibility of FACETs concepts. SBE scientists specializing in human factors, psychology, sociology, and communication, for example, are guiding studies on forecaster interaction with probabilistic data, public risk awareness/response, and risk communication. Likewise, key stakeholders from media, commercial weather enterprises, and emergency management are involved in FACETs development and evaluation. The successful implementation of the paradigm will require bold policy decisions, close stakeholder interactions, physical and SBE science advances, and weather enterprise innovation.

SUMMARY.

FACETs is a comprehensive and potentially transformational hazard-forecasting paradigm being developed by NOAA in response to recommendations of several distinguished scientific bodies (e.g., NRC, NIST, and WRN). Probabilistic hazard information (PHI) forecasting is at the heart of the FACETs paradigm, and the delivery of continuous, rapidly updated PHI is expected to fill gaps in the existing, deterministic watch–warning paradigm. In so doing, FACETs may provide significant opportunities to address the recommendations of the scientific community and enhance the nation’s hazardous weather forecasting enterprise.

The authors readily acknowledge uncertainties regarding FACETs’ postimplementation effectiveness in reducing loss of life and property over that found in the current, legacy paradigm. Further, many FACETs aspects are yet to be fully explored and developed, importantly including a full conceptual typology of the opportunities and challenges that may be posed by introducing a continuous stream of PHI with a variety of overlapping, intersecting but unique reference classes. We believe, however, there is sufficient anecdotal, experiential, and community-driven evidence to anticipate gap-filling, probability-based information will be of greater value to society than the current, deterministic paradigm. Our intention is to determine this value through applied physical and SBE sciences in preimplementation testing and constant, regular, and focused postimplementation evaluations. Those efforts must occur in full collaboration with members of commercial weather enterprises and the diverse populations who will receive and act upon the probabilistic information.

FACETs, on the whole, is intended to be a clear and unifying vision, framework, and concept that i) guides research and development work; ii) accommodates relevant trajectories of science, technology, and societal needs; and iii) fosters weather enterprise partnerships to enable next-generation improvements in the United States’ hazardous weather forecasting system. The ultimate goal of FACETs is to minimize the impact of all hazardous weather on society and support a truly Weather-Ready Nation.

ACKNOWLEDGMENTS

The authors are grateful to Mr. Charles Kuster for his literature search supporting this publication and to the reviewers of this work for their helpful recommendations. A portion of the funding for this work was provided by NOAA/Office of Oceanic and Atmospheric Research under NOAA–University of Oklahoma Cooperative Agreement NA11OAR4320072, U.S. Department of Commerce. The statements, findings, conclusions, and recommendations are those of the authors and do not necessarily reflect the views of NOAA or the U.S. Department of Commerce.

REFERENCES

REFERENCES
Argyle
,
E.
,
J.
Gourley
,
Z.
Flamig
,
T.
Hansen
, and
K.
Manross
,
2017
:
Toward a user-centered design of a weather forecasting decision-support tool
.
Bull. Amer. Meteor. Soc.
,
98
,
373
382
, https://doi.org/10.1175/BAMS-D-16-0031.1.
Ash
,
K. D.
,
R. L.
Schumann
III
, and
G. C.
Bowser
,
2014
:
Tornado warning trade-offs: Evaluating choices for visually communicating risk
.
Wea. Climate Soc.
,
6
,
104
118
, https://doi.org/10.1175/WCAS-D-13-00021.1.
Barthold
,
F.
,
T.
Workoff
,
B.
Cosgrove
,
J.
Gourley
,
D.
Novak
, and
K.
Mahoney
,
2015
:
Improving flash flood forecasts: The HMT-WPC Flash Flood and Intense Rainfall Experiment
.
Bull. Amer. Meteor. Soc.
,
96
,
1859
1866
, https://doi.org/10.1175/BAMS-D-14-00201.1.
Benjamin
,
S. G.
, and Coauthors
,
2016
:
A North American hourly assimilation and model forecast cycle: The Rapid Refresh
.
Mon. Wea. Rev.
,
144
,
1669
1694
, https://doi.org/10.1175/MWR-D-15-0242.1.
Broad
,
K.
,
A.
Leiserowitz
,
J.
Weinkle
, and
M.
Steketee
,
2007
:
Misinterpretations of the “cone of uncertainty” in Florida during the 2004 hurricane season
.
Bull. Amer. Meteor. Soc.
,
88
,
651
667
, https://doi.org/10.1175/BAMS-88-5-651.
Brotzge
,
J.
, and
W.
Donner
,
2013
:
The tornado warning process: A review of current research, challenges, and opportunities
.
Bull. Amer. Meteor. Soc.
,
94
,
1715
1733
, https://doi.org/10.1175/BAMS-D-12-00147.1.
Brown
,
L. M.
,
D. M.
Dosa
,
K.
Thomas
,
K.
Hyer
,
Z.
Feng
, and
V.
Mor
,
2012
:
The effects of evacuation on nursing home residents with dementia
.
Amer. J. Alzheimers Dis. Other Dementias
,
27
,
406
412
, https://doi.org/10.1177/1533317512454709.
Bunting
,
W.
,
2015
: An update on Storm Prediction Center risk communication strategies and future services. 41st National Weather Association Annual Meeting/Broadcaster Workshop,
Norfolk, VA
,
National Weather Association
, www.nwas.org/meetings/nwas16/agenda.html.
Casteel
,
M.
,
2016
:
Communicating increased risk: An empirical investigation of the National Weather Service’s impact-based warnings
.
Wea. Climate Soc.
,
8
,
219
232
, https://doi.org/10.1175/WCAS-D-15-0044.1.
Casteel
,
M.
, and
J.
Downing
,
2013
:
How individuals process NWS weather warning messages on their cell phones
.
Wea. Climate Soc.
,
5
,
254
265
, https://doi.org/10.1175/WCAS-D-12-00031.1.
Cintineo
,
J.
,
T.
Smith
,
V.
Lakshmanan
,
H.
Brooks
, and
K.
Ortega
,
2012
:
An objective high-resolution hail climatology of the contiguous United States
.
Wea. Forecasting
,
27
,
1235
1248
, https://doi.org/10.1175/WAF-D-11-00151.1.
Cintineo
,
J.
,
M. J.
Pavolonis
,
J. M.
Sieglaff
, and
D. T.
Lindsey
,
2014
:
An empirical model for assessing the severe weather potential of developing convection
.
Wea. Forecasting
,
29
,
639
653
, https://doi.org/10.1175/WAF-D-13-00113.1.
Coleman
,
T.
,
K.
Knupp
,
J.
Spann
,
J.
Elliott
, and
B.
Peters
,
2011
:
The history (and future) of tornado warning dissemination in the United States
.
Bull. Amer. Meteor. Soc.
,
92
,
567
582
, https://doi.org/10.1175/2010BAMS3062.1.
Dance
,
S.
,
E.
Elbert
, and
D.
Scurrah
,
2010
:
Thunderstorm strike probability nowcasting
.
J. Atmos. Oceanic Technol.
,
27
,
79
93
, https://doi.org/10.1175/2009JTECHA1279.1.
Demuth
,
J. L.
,
E.
Gruntfest
,
R. E.
Morss
,
S.
Drobot
, and
J. K.
Lazo
,
2007
:
WAS*IS: Building a community for integrating meteorology and social science
.
Bull. Amer. Meteor. Soc.
,
88
,
1729
1737
, https://doi.org/10.1175/BAMS-88-11-1729.
Donner
,
W. R.
,
H.
Rodriguez
, and
W.
Diaz
,
2012
:
Tornado warnings in three southern states: A qualitative analysis of public response patterns
.
J. Homeland Secur. Emerg. Manage.
,
9
, https://doi.org/10.1515/1547-7355.1955.
Doswell
,
C. A.
, III
,
1992
:
Forecaster workstation design: Concepts and issues
.
Wea. Forecasting
,
7
,
398
407
, https://doi.org/10.1175/1520-0434(1992)007<0398:FWDCAI>2.0.CO;2.
Doswell
,
C. A.
,
2004
:
Weather forecasting by humans—Heuristics and decision making
.
Wea. Forecasting
,
19
,
1115
1126
, https://doi.org/10.1175/WAF-821.1.
Doswell
,
C. A.
,
A.
Moller
, and
H.
Brooks
,
1999
:
Storm spotting and public awareness since the first tornado forecasts of 1948
.
Wea. Forecasting
,
14
,
544
557
, https://doi.org/10.1175/1520-0434(1999)014<0544:SSAPAS>2.0.CO;2.
Droegemeier
,
K. D.
,
L. P.
Rothfusz
,
A. J.
Knoedler
,
J. T.
Ferree
,
J.
Henderson
,
K. L.
Nemunaitis-Monroe
,
D.
Nagele
, and
K. E.
Klockow
,
2016
: Living with Extreme Weather Workshop: Summary and path forward. 11th Symp. on Societal Applications: Policy, Research and Practice, New Orleans, LA, Amer. Meteor. Soc., 9.1, https://ams.confex.com/ams/96Annual/webprogram/Paper290837.html.
Edwards
,
R.
, and
F.
Ostby
,
2009
: Time line of SELS and SPC. NOAA/SPC, www.spc.noaa.gov/history/timeline.html.
Ferree
,
J. T.
,
D.
Freeman
,
E.
Jacks
, and
J. M.
Looney
,
2007
:
Storm-based warnings changes to NWS warnings for the Digital Age
. 35th Conf. on Broadcast Meteorology, San Antonio, TX, Amer. Meteor. Soc., 5.6., https://ams.confex.com/ams/87ANNUAL/techprogram/paper_120818.htm.
FGDC
,
2001
: United States National Grid. Federal Geographic Data Committee Rep. FGDC-STD-011-2001, 39 pp., www.fgdc.gov/usng/fgdc_std_011_2001_usng.pdf.
Florida Health Care Education and Development Foundation
,
2008
: National criteria for evacuation decision-making in nursing homes. John A. Hartford Foundation Rep., 17 pp., www.hcanj.org/files/2013/09/National-Criteria-for-Evacuation-Decision-making-in-Nursing-Facilities-FHCA-May-2008.pdf.
Frederick
,
J.
, and
S.
Amburn
,
2015
:
Bridging the watch to warning DSS gap with lightning derived petals
. Seventh Conf. on the Meteorological Applications of Lightning Data, Phoenix, AZ, Amer. Meteor. Soc., 5.3, https://ams.confex.com/ams/95Annual/webprogram/Paper265316.html.
Galluppi
,
K.
,
S. F.
Piltz
,
K.
Nuckles
,
B.
Montz
,
J.
Correia
, and
R.
Riley
,
2015
: Improving weather and emergency management messaging: The Tulsa Weather Message Experiment. 10th Symp. on Societal Applications: Policy, Research and Practice, Phoenix, AZ, Amer. Meteor. Soc., 3, https://ams.confex.com/ams/95Annual/webprogram/Paper270086.html.
Gilbert
,
K. K.
,
J. P.
Craven
,
D. R.
Novak
,
T. M.
Hamill
,
J.
Sieveking
,
D. P.
Ruth
, and
S. J.
Lord
,
2015
: An introduction to the National Blend of Global Models Project. Special Symp. on Model Postprocessing and Downscaling, Phoenix, AZ, Amer. Meteor. Soc., 3.1, https://ams.confex.com/ams/95Annual/webprogram/Paper267282.html.
Gladwin
,
H.
,
H.
Willoughby
,
J.
Lazo
,
B.
Morrow
, and
W.
Peacock
,
2009
:
Social science research needs for the Hurricane Forecast and Warning System
.
Bull. Amer. Meteor. Soc.
,
90
,
25
29
, https://doi.org/10.1175/2008BAMS2606.1.
Glahn
,
H. R.
, and
D. P.
Ruth
,
2003
:
The new digital forecast database of the National Weather Service
.
Bull. Amer. Meteor. Soc.
,
84
,
195
201
, https://doi.org/10.1175/BAMS-84-2-195.
Golden
,
J. H.
, and
C. R.
Adams
,
2000
:
The tornado problem: Forecast, warning, and response
.
Nat. Hazards Rev.
,
1
,
107
118
, https://doi.org/10.1061(ASCE)1527-6988(2000)1:2(107).
Gruntfest
,
E.
, and
H.
Lazrus
,
2009
:
Social Science Woven into Meteorology (SSWIM) strategic plan. Weaving social science into weather and climate research & practice: A living roadmap
.
CIMMS Rep.
,
4
pp., http://cimms.ou.edu/sswim/pdfs/plan.pdf.
Hamill
,
T.
,
1997
:
Reliability diagrams for multicategory probabilistic forecasts
.
Wea. Forecasting
,
12
,
736
741
, https://doi.org/10.1175/1520-0434(1997)012<0736:RDFMPF>2.0.CO;2.
Hansen
,
T.
,
T.
Dankers
, and
C.
Paxton
,
2003
: Text formatting with the Graphical Forecast Editor. 19th Int. Conf. on Interactive Information Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, Long Beach, CA, Amer. Meteor. Soc., P1.11, https://ams.confex.com/ams/pdfpapers/54419.pdf.
Hirschberg
,
P.
, and Coauthors
,
2011
:
A weather and climate enterprise strategic implementation plan for generating and communicating forecast uncertainty information
.
Bull. Amer. Meteor. Soc.
,
92
,
1651
1666
, https://doi.org/10.1175/BAMS-D-11-00073.1.
Hoekstra
,
S.
,
K.
Klockow
,
R.
Riley
,
J.
Brotzge
,
H.
Brooks
, and
S.
Erickson
,
2011
:
A preliminary look at the social perspective of Warn-on-Forecast: Preferred tornado warning lead time and the general public’s perceptions of weather risks
.
Wea. Climate Soc.
,
3
,
128
140
, https://doi.org/10.1175/2011WCAS1076.1.
Hoffman
,
R. R.
,
1991
:
Human factors psychology in the support of forecasting: The design of advanced meteorological workstations
.
Wea. Forecasting
,
6
,
98
110
, https://doi.org/10.1175/1520-0434(1991)006<0098:HFPITS>2.0.CO;2.
Howe
,
P. D.
,
H.
Boudet
,
A.
Leiserowitz
, and
E. W.
Maibach
,
2014
:
Mapping the shadow of experience in extreme weather events
.
Climatic Change
,
127
,
381
389
, https://doi.org/10.1007/s10584-014-1253-6.
Jacks
,
E.
, and
J.
Ferree
,
2007
: Socio-economic impacts of storm-based warnings. Second Symp. on Policy and Socio-Economic Impacts, San Antonio, TX, Amer. Meteor. Soc., 2.2, https://ams.confex.com/ams/87ANNUAL/techprogram/paper_120849.htm.
Jacks
,
E.
, and Coauthors
,
2013
: Improving NWS communication: Hazard simplification demonstration. Second Conf. on Weather Warnings and Communication, Nashville, TN, Amer. Meteor. Soc., J3.3, https://ams.confex.com/ams/41BC2WxWarn/webprogram/Paper225780.html.
Joslyn
,
S.
, and
J.
LeClerc
,
2013
:
Decisions with uncertainty: The glass half full
.
Curr. Dir. Psychol. Sci.
,
22
,
308
315
, https://doi.org/10.1177/0963721413481473.
Joslyn
,
S.
,
K.
Pak
,
D.
Jones
,
J.
Pyles
, and
E.
Hunt
,
2007
:
The effect of probabilistic information of threshold forecasts
.
Wea. Forecasting
,
22
,
804
812
, https://doi.org/10.1175/WAF1020.1.
Karstens
,
C. D.
, and Coauthors
,
2015
:
Evaluation of a probabilistic forecasting methodology for severe convective weather in the 2014 Hazardous Weather Testbed
.
Wea. Forecasting
,
30
,
1551
1570
, https://doi.org/10.1175/WAF-D-14-00163.1.
Karstens
,
C. D.
, and Coauthors
,
2016
: Forecaster decision-making with automated probabilistic guidance in the 2015 Hazardous Weather Testbed Probabilistic Hazard Information Experiment. Fourth Symp. on Building a Weather-Ready Nation: Enhancing Our Nation’s Readiness, Responsiveness, and Resilience to High Impact Weather Events, New Orleans, LA, Amer. Meteor. Soc., 4.2, https://ams.confex.com/ams/96Annual/webprogram/Paper286854.html.
Kay
,
M. P.
, and
H. E.
Brooks
,
2000
:
Verification of probabilistic severe storm forecasts at the SPC. 20th Conf. on Severe Local Storms
,
Orlando, FL
,
Amer. Meteor. Soc.
, 9.3, https://ams.confex.com/ams/Sept2000/techprogram/paper_15921.htm.
Kuhlman
,
K. M.
,
T. M.
Smith
,
G. J.
Stumpf
,
K. L.
Ortega
, and
K. L.
Manross
,
2008
:
Experimental probabilistic hazard information in practice: Results from the 2008 EWP Spring Program
. 24th Conf. on Severe Local Storms,
Savannah, GA
,
Amer. Meteor. Soc.
, 8A.2, https://ams.confex.com/ams/24SLS/techprogram/paper_142027.htm.
Kuhlman
,
K. M.
,
E.
Gruntfest
,
K. A.
Scharfenberg
, and
G. J.
Stumpf
,
2009
: Beyond storm-based warnings: An advanced WAS*IS workshop to study communication of probabilistic hazardous weather information. Fourth Symp. on Policy and Socio-Economic Research, Phoenix, AZ, Amer. Meteor. Soc., 3.5, https://ams.confex.com/ams/89annual/techprogram/paper_150887.htm.
LaDue
,
D.
,
S.
Ernst
,
C. D.
Karstens
,
J.
Corriea
Jr.
,
J. E.
Hocker
, and
J. P.
Wolfe
,
2016
: Co-creating the form and function of the prototype probabilistic hazard information (PHI) to meet emergency manager user needs. 11th Symp. on Societal Applications: Policy, Research and Practice, New Orleans, LA, Amer. Meteor. Soc., 7.4, https://ams.confex.com/ams/96Annual/webprogram/Paper280331.html.
Lazo
,
J.
,
2010
:
The costs and losses of integrating social sciences and meteorology
.
Wea. Climate Soc.
,
2
,
171
173
, https://doi.org/10.1175/2010WCAS1086.1.
Lazo
,
J.
,
R.
Morss
, and
J.
Demuth
,
2009
:
300 billion served: Sources, perceptions, uses, and values of weather forecasts
.
Bull. Amer. Meteor. Soc.
,
90
,
785
798
, https://doi.org/10.1175/2008BAMS2604.1.
League
,
C. E.
,
W.
Díaz
,
B.
Philips
,
E. J.
Bass
,
K.
Kloesel
,
E.
Gruntfest
, and
A.
Gessner
,
2010
:
Emergency manager decision-making and tornado warning communication
.
Meteor. Appl.
,
17
,
163
172
, https://doi.org/10.1002/met.201.
LeClerc
,
J.
, and
S.
Joslyn
,
2012
:
Odds ratio forecasts increase precautionary action for extreme weather events
.
Wea. Climate Soc.
,
4
,
263
270
, https://doi.org/10.1175/WCAS-D-12-00013.1.
Lindell
,
M. K.
, and
H.
Brooks
,
2012
: Workshop on Weather Ready Nation: Science imperatives for severe thunderstorm research, held 24–26 April 2012 in Birmingham, AL. Hazard Reduction and Recovery Center Research Rep. 12-02R,
Texas A&M University
,
5
pp., http://hrrc.arch.tamu.edu/_common/documents/Workshop%20on%20Weather%20Ready%20Nation_2.pdf.
Lindell
,
M. K.
, and
H.
Brooks
,
2013
:
Workshop on Weather Ready Nation: Science imperatives for severe thunderstorm research
.
Bull. Amer. Meteor. Soc.
,
94
,
ES171
ES174
, https://doi.org/10.1175/BAMS-D-12-00238.1.
Lindell
,
M. K.
,
S.
Huang
,
H.
Wei
, and
C. D.
Samuelson
,
2016
:
Perceptions and expected immediate reactions to tornado warning polygons
.
Nat. Hazards
,
80
,
683
707
, https://doi.org/10.1007/s11069-015-1990-5.
Ling
,
C.
,
L.
Hua
,
C. D.
Karstens
,
G. J.
Stumpf
,
T. M.
Smith
,
K. M.
Kuhlman
, and
L.
Rothfusz
,
2015
.
A comparison between WarnGen system and probabilistic hazard information system for severe weather forecasting
.
Proc. Hum. Factors Ergon. Soc. Annu. Meet.
,
59
,
1791
1795
, https://doi.org/10.1177/1541931215591387.
Lusk
,
C.
,
P.
Kucera
,
W.
Roberts
, and
L.
Johnson
,
1999
:
The process and methods used to evaluate prototype operational hydrometeorological workstations
.
Bull. Amer. Meteor. Soc.
,
80
,
57
65
, https://doi.org/10.1175/1520-0477(1999)080<0057:TPAMUT>2.0.CO;2.
McGovern
,
A.
,
C. D.
Karstens
,
T. M.
Smith
, and
K. M.
Kuhlman
,
2016
:
Using machine learning for nowcasting severe weather hazards
. 14th Conf. on Artificial and Computational Intelligence and Its Applications to the Environmental Sciences,
New Orleans, LA
,
Amer. Meteor. Soc.
, 3.6, https://ams.confex.com/ams/96Annual/webprogram/Paper290541.html.
Mileti
,
D. S.
, and
J. H.
Sorensen
,
1990
:
Communication of emergency public warnings: A social science perspective and state-of-the-art assessment. FEMA Doc
. ORNL 6609,
166
pp., www.cires.org.mx/docs_info/CIRES_003.pdf.
Miran
,
S. M.
,
C.
Ling
,
J.
James
,
A. E.
Gerard
, and
L. P.
Rothfusz
,
2017
:
User perception and interpretation of tornado probabilistic hazard information: Comparison of four graphical designs
.
Appl. Ergon.
,
65
,
277
285
, https://doi.org/10.1016/j.apergo.2017.06.016.
Mitchell
,
M. J.
,
2014
: Probabilistic tornado warning plumes. 27th Conf. on Severe Local Storms,
Madison, WI
,
Amer. Meteor. Soc.
, P3.38, https://ams.confex.com/ams/27SLS/webprogram/Paper254286.html.
Morrow
,
B.
,
J.
Lazo
,
J.
Rhome
, and
J.
Feyen
,
2015
:
Improving storm surge risk communication: Stakeholder perspectives
.
Bull. Amer. Meteor. Soc.
,
96
,
35
48
, https://doi.org/10.1175/BAMS-D-13-00197.1.
Morss
,
R. E.
,
J. L.
Demuth
, and
J. K.
Lazo
,
2008
:
Communicating uncertainty in weather forecasts: A survey of the U.S. public
.
Wea. Forecasting
,
23
,
974
991
, https://doi.org/10.1175/2008WAF2007088.1.
Morss
,
R. E.
,
J. K.
Lazo
, and
J. L.
Demuth
,
2010
:
Examining the use of weather forecasts in decision scenarios: Results from a US survey with implications for uncertainty communication
.
Meteor. Appl.
,
17
,
149
162
, https://doi.org/10.1002/met.196.
Morss
,
R. E.
,
J. K.
Lazo
,
J. L.
Demuth
,
K.
Dickinson
,
H.
Lazrus
, and
B.
Morrow
,
2016
:
Understanding public hurricane evacuation decisions and responses to forecast and warning messages
.
Wea. Forecasting
,
31
,
395
417
, https://doi.org/10.1175/WAF-D-15-0066.1.
Morss
,
R. E.
, and Coauthors
,
2017
:
Hazardous weather prediction and communication in the modern information environment
.
Bull. Amer. Meteor. Soc.
,
98
,
2653
2674
, https://doi.org/10.1175/BAMS-D-16-0058.1.
Murphy
,
A.
,
1991
:
Probabilities, odds, and forecasts of rare events
.
Wea. Forecasting
,
6
,
302
307
, https://doi.org/10.1175/1520-0434(1991)006<0302:POAFOR>2.0.CO;2.
Murphy
,
A.
,
1993
:
What is a good forecast? An essay on the nature of goodness in weather forecasting
.
Wea. Forecasting
,
8
,
281
293
, https://doi.org/10.1175/1520-0434(1993)008<0281:WIAGFA>2.0.CO;2.
Nagele
,
D. E.
, and
J. E.
Trainor
,
2012
:
Geographic specificity, tornadoes, and protective action
.
Wea. Climate Soc.
,
4
,
145
155
, https://doi.org/10.1175/WCAS-D-11-00047.1.
National Academy of Public Administration
,
2013
: Forecast for the future: Assuring the capacity of the National Weather Service.
National Academy of Public Administration Publ.
,
137
pp., www.napawash.org/uploads/Academy_Studies/ForecastfortheFuture-AssuringtheCapacityoftheNationalWeatherService.pdf.
Niedermoser
,
B.
,
M.
Ungersbock
,
A.
Ohms
, and
M.
Staudinger
,
2007
:
EMMA/METEOALARM—European warnings
.
ECMWF Rep.
,
89
pp., www.ecmwf.int/sites/default/files/elibrary/2008/12390-european-meteorological-warnings-eumetnet-meteoalarm-system.pdf.
NIST
,
2013
: Technical investigation of the May 22, 2011 tornado in Joplin,
Missouri
.
National Institute of Standards and Technology Final Rep.
,
428
pp., www.nist.gov/publications/final-report-national-institute-standards-and-technology-nist-technical-investigation.
Novak
,
D. R.
,
D. R.
Bright
, and
M. J.
Brennan
,
2008
:
Operational forecaster uncertainty needs and future roles
.
Wea. Forecasting
,
23
,
1069
1084
, https://doi.org/10.1175/2008WAF2222142.1.
Novak
,
D. R.
,
K.
Brill
, and
W.
Hogsett
,
2014
:
Using percentiles to communicate snowfall uncertainty
.
Wea. Forecasting
,
29
,
1259
1265
, https://doi.org/10.1175/WAF-D-14-00019.1.
NRC
,
1999
: A Vision for the National Weather Service: Road Map of the Future.
National Academies Press
,
88
pp.
NRC
,
2003
: Communicating Uncertainties in Weather and Climate Information.
National Academies Press
,
68
pp.
NRC
,
2006
: Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts.
National Academies Press
,
122
pp., https://doi.org/10.17226/11699.
NRC
,
2012a
: The National Weather Service Modernization and Associated Restructuring: A Retrospective Assessment.
National Academies Press
,
121
pp.
NRC
,
2012b
: Weather Services for the Nation: Becoming Second to None.
National Academies Press
,
88
pp.
NRC
,
2013
: Environmental Decisions in the Face of Uncertainty.
National Academies Press
,
281
pp.
NWS
,
1994
:
Superstorm of March 1993
.
NOAA National Disaster Survey Rep.
,
145
pp., www.weather.gov/media/publications/assessments/Superstorm_March-93.pdf.
NWS
,
1999
:
Oklahoma/southern Kansas tornado outbreak of May 3, 1999
.
NOAA National Disaster Survey Rep.
,
51
pp., www.weather.gov/media/publications/assessments/ok-ks99.pdf].
NWS
,
2009
:
Central United States flooding of June 2008
.
NOAA National Disaster Survey Rep.
,
104
pp., www.weather.gov/media/publications/assessments/central_flooding09.pdf.
NWS
,
2011a
: The historic tornadoes of April 2011.
NOAA National Disaster Survey Rep.
,
76
pp., www.weather.gov/media/publications/assessments/historic_tornadoes.pdf.
NWS
,
2011b
: Joplin, Missouri tornado–May 22, 2011.
NWS Central Region Service Assessment
,
35
pp., www.weather.gov/media/publications/assessments/Joplin_tornado.pdf.
NWS
,
2012
:
WFO severe weather products specification
.
National Weather Service Instruction
10-511
,
35
pp., www.nws.noaa.gov/directives/sym/pd01005011curr.pdf.
NWS
,
2014
:
May 2013 Oklahoma tornadoes and flash flooding
.
NWS Service Assessment
,
63
pp., www.weather.gov/media/publications/assessments/13oklahoma_tornadoes.pdf.
Ortega
,
K. L.
,
2015
: A radar-based storm climatology for the contiguous United States for improved severe weather climatologies and warnings. 31st Conf. on Environmental Information Processing Technologies,
Phoenix, AZ
,
Amer. Meteor. Soc.
, 10.3, https://ams.confex.com/ams/95Annual/webprogram/Paper262017.html.
Palmer
,
T. D.
, and Coauthors
,
2013
: Implementation of enhanced short term and aviation grids at the NWS Atlanta Forecast Office. 16th Conf. on Aviation, Range, and Aerospace Meteorology,
Austin, TX
,
Amer. Meteor. Soc.
, J1.2, https://ams.confex.com/ams/93Annual/webprogram/Paper216125.html.
Polger
,
P. D.
,
B. S.
Goldsmith
,
R. C.
Przywarty
, and
J. R.
Bocchieri
,
1994
:
National Weather Service warning performance based on the WSR-88D
.
Bull. Amer. Meteor. Soc.
,
75
,
203
214
, https://doi.org/10.1175/1520-0477(1994)075<0203:NWSWPB>2.0.CO;2.
Reeves
,
R.
,
J.
Tate
, and
A.
Treadway
,
2013
: A comparative analysis of severe weather warnings in the continental United States.
University of Oklahoma School of Meteorology Capstone Program Rep.
,
31
pp., http://weather.ou.edu/∼warnstats/Warnstats.pdf.
Ripberger
,
J.
,
C.
Silva
,
H.
Jenkins-Smith
, and
M.
James
,
2015
:
The influence of consequence-based messages on public responses to tornado warnings
.
Bull. Amer. Meteor. Soc.
,
96
,
577
590
, https://doi.org/10.1175/BAMS-D-13-00213.1.
Roberts
,
R.
,
A.
Anderson
,
E.
Nelson
,
B.
Brown
,
J.
Wilson
,
M.
Pocernich
, and
T.
Saxen
,
2012
:
Impacts of forecaster involvement on convective storm initiation and evolution nowcasting
.
Wea. Forecasting
,
27
,
1061
1089
, https://doi.org/10.1175/WAF-D-11-00087.1.
Rothfusz
,
L. P.
, and Coauthors
,
2014a
:
Forecasting a Continuum of Environmental Threats (FACETs) Science and Strategic Implementation Plan (SSIP)
.
SSIP Development Team Rep.
,
88
pp., www.nssl.noaa.gov/projects/facets/FACETs%20SSIP_1.pdf.
Rothfusz
,
L. P.
,
C.
Karstens
, and
D.
Hilderbrand
,
2014b
:
Next-generation severe weather forecasting and communication
.
Eos, Trans. Amer. Geophys. Union
,
95
,
325
326
, https://doi.org/10.1002/2014EO360001.
Rothman
,
M.
, and
L. M.
Brown
,
2007
:
The vulnerable geriatric casualty: Medical needs of frail older adults during disasters
.
Generations
,
31
(
4
),
20
24
.
Sanders
,
F.
,
1963
:
On subjective probability forecasting
.
J. Appl. Meteor.
,
2
,
191
201
, https://doi.org/10.1175/1520-0450(1963)002<0191:OSPF>2.0.CO;2.
Scharfenberg
,
K.
,
J.
Crockett
,
J.
Keyes
,
M. W.
Davis
,
W. M.
Thomas
, and
L.
Dunn
,
2015
: The National Weather Service Impacts Catalog. Third Symp. on Building a Weather-Ready Nation: Enhancing Our Nation’s Readiness, Responsiveness, and Resilience to High Impact Weather Events, Phoenix, AZ,
Amer. Meteor. Soc.
, 6.3, https://ams.confex.com/ams/95Annual/webprogram/Paper265097.html.
Schumacher
,
R. S.
,
D. T.
Lindsey
,
A. B.
Schumacher
,
J.
Braun
,
S. D.
Miller
, and
J. L.
Demuth
,
2010
:
Multidisciplinary analysis of an unusual tornado: Meteorology, climatology, and the communication and interpretation of warnings
.
Wea. Forecasting
,
25
,
1412
1429
, https://doi.org/10.1175/2010WAF2222396.1.
Senkbeil
,
J. C.
,
D. A.
Scott
,
P.
Guinazu-Walker
, and
M. S.
Rockman
,
2013
:
Ethnic and racial differences in tornado hazard perception, preparedness, and shelter lead time in Tuscaloosa
.
Prof. Geogr.
,
66
,
610
620
, https://doi.org/10.1080/00330124.2013.826562.
Sheets
,
R. C.
,
1985
:
The National Weather Service Hurricane Probability Program
.
Bull. Amer. Meteor. Soc.
,
66
,
4
13
, https://doi.org/10.1175/1520-0477(1985)066<0004:TNWSHP>2.0.CO;2.
Simmons
,
K. M.
, and
D.
Sutter
,
2005
:
WSR-88D radar, tornado warnings, and tornado casualties
.
Wea. Forecasting
,
20
,
301
310
, https://doi.org/10.1175/WAF857.1.
Simmons
,
K. M.
, and
D.
Sutter
,
2008
:
Tornado warnings, lead times, and tornado casualties: An empirical investigation
.
Wea. Forecasting
,
23
,
246
258
, https://doi.org/10.1175/2007WAF2006027.1.
Smith
,
T.
, and Coauthors
,
2016
:
Multi-Radar Multi-Sensor (MRMS) severe weather and aviation products: Initial operating capabilities
.
Bull. Amer. Meteor. Soc.
,
97
,
1617
1630
, https://doi.org/10.1175/BAMS-D-14-00173.1.
Snook
,
N.
,
Y.
Jung
,
J.
Brotzge
,
B.
Putnam
, and
M.
Xue
,
2016
:
Prediction and ensemble forecast verification of hail in the supercell storms of 20 May 2013. Wea
.
Forecasting
,
31
,
811
825
, https://doi.org/10.1175/WAF-D-15-0152.1.
Sobash
,
R. A.
,
J. S.
Kain
,
D. R.
Bright
,
A. R.
Dean
,
M. C.
Coniglio
, and
S. J.
Weiss
,
2011
:
Probabilistic forecast guidance for severe thunderstorms based on the identification of extreme phenomena in convection-allowing model forecasts
.
Wea. Forecasting
,
26
,
714
728
, https://doi.org/10.1175/WAF-D-10-05046.1.
Stensrud
,
D. J.
, and Coauthors
,
2009
:
Convective-scale Warn-on-Forecast system: A vision for 2020
.
Bull. Amer. Meteor. Soc.
,
90
,
1487
1499
, https://doi.org/10.1175/2009BAMS2795.1.
Stumpf
,
G. J.
,
T. M.
Smith
,
K. L.
Manross
, and
D. L.
Andra
Jr.
,
2008
: The Experimental Warning Program 2008 Spring Experiment at the NOAA Hazardous Weather Testbed. 24th Conf. on Severe Local Storms,
Savannah, GA
,
Amer. Meteor. Soc.
, 8A.1, http://ams.confex.com/ams/24SLS/techprogram/paper_141712.htm.
Stumpf
,
G. J.
,
C. D.
Karstens
, and
L. P.
Rothfusz
,
2015
: Probabilistic hazard information (PHI): Highlighting the benefits via new verification techniques for FACETs. Third Conf. on Weather Warnings and Communication,
Raleigh, NC
,
Amer. Meteor. Soc.
, 5.7, https://ams.confex.com/ams/43BC3WxWarn/webprogram/Paper272745.html.
Sutter
,
D.
, and
S.
Erickson
,
2010
:
The time cost of tornado warnings and the savings with storm-based warnings
.
Wea. Climate Soc.
,
2
,
103
112
, https://doi.org/10.1175/2009WCAS1011.1.
Trainor
,
J.
,
D.
Nagele
,
B.
Philips
, and
B.
Scott
,
2015
:
Tornadoes, social science, and the false alarm effect
.
Wea. Climate Soc.
,
7
,
333
352
, https://doi.org/10.1175/WCAS-D-14-00052.1.
Waldstreicher
,
J. S.
,
B. A.
Smith
,
J.
Dellicarpini
,
M.
Belk
,
H.
Gerapetritis
, and
C.
Leonardi
,
2012
: Integrating digital aviation services into forecast operations. Third Aviation, Range, and Aerospace Meteorology Special Symp. on Weather-Air Traffic Management Integration,
New Orleans, LA
,
Amer. Meteor. Soc
., https://ams.confex.com/ams/92Annual/webprogram/Paper196714.html.
Waldstreicher
,
J. S.
,
D. B.
Radell
,
K. W.
Johnson
,
J.
Guiney
, and
R.
Watling
,
2017
: The probabilistic storm total snowfall forecast experiment: 2015–16 results and future plans. 28th Conf. on Weather Analysis and Forecasting,
Seattle, WA
,
Amer. Meteor. Soc.
, 4A.4, https://ams.confex.com/ams/97Annual/webprogram/Paper308374.html.
Wang
,
X. H.
, and
N.
Kapucu
,
2008
:
Public complacency under repeated emergency threats: Some empirical evidence
.
J. Public Adm. Res. Theory
,
18
,
57
78
, https://doi.org/10.1093/jopart/mum001.
Yussouf
,
N.
,
D.
Dowell
,
L.
Wicker
,
K.
Knopfmeier
, and
D.
Wheatley
,
2015
:
Storm-scale data assimilation and ensemble forecasts for the 27 April 2011 severe weather outbreak in Alabama
.
Mon. Wea. Rev.
,
143
,
3044
3066
, https://doi.org/10.1175/MWR-D-14-00268.1.
Zhang
,
J.
, and Coauthors
,
2016
:
Multi-Radar Multi-Sensor (MRMS) quantitative precipitation estimation: Initial operating capabilities
.
Bull. Amer. Meteor. Soc.
,
97
,
621
638
, https://doi.org/10.1175/BAMS-D-14-00174.1.

Footnotes

For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).