Many coastal communities are affected by extreme precipitation, wind, and storm surge flooding, and the impacts are amplified by dense population, local industry, and proximity to marine ecosystems. Almost 40% of the U.S. population lives near coastlines (NOAA National Ocean Service 2013). The exacerbating effect of climate change on weather (Walsh et al. 2014) requires both individual-protective and collective decision-making. Growing recognition of the importance of human responses in the face of extreme weather forecasts has resulted in new decision support systems such as impact-based decision support services, now an integral part of National Weather Service operations (Uccellini and Ten Hoeve 2019). Forecasters need to communicate with the public in ways that encourage appropriate action, which often requires additional engagement from emergency managers and community stakeholders.
Several factors influence individuals’ risk perceptions and willingness to take action that protects life and property. These include individual, familial, and socioeconomic situations; evacuation ability; and information source, quality, and delivery (Burnside et al. 2007; Freeman et al. 2019). In the face of uncertainty, people have difficulty assessing risk to inform decision-making (Brust-Renck et al. 2014; Simis et al. 2016; Toplak et al. 2014; Tversky and Kahneman 1992).
Previous research has explored how visualizations influence perceptions and behaviors related to preparing for and responding to natural hazards (Mol et al. 2022; Simpson et al. 2022). Immersive virtual reality (i.e., technology that replaces real-world surroundings and allows users to engage with a created environment) has been used to train emergency responders (Lipp et al. 2021) and relief workers (Eva and Ladislav 2017), and to investigate evacuation-related behavior (Freeman et al. 2019; Fu et al. 2021). To our knowledge, little research has focused on the use of visualization to help students consider the need to take protective action in the face of severe weather.
Storm evacuation and preparedness may also depend on communal factors. The ethical matrix is a method developed by philosophers of bioethics to represent perspectives of stakeholders within a sphere of decision-making (Mepham et al. 2006). It uses a visual representation of stakeholders’ values to identify, discuss, and document conflicting individual perspectives and societal trade-offs. The ethical matrix has been used in contentious policy domains to build consensus (Forsberg 2004; Mepham et al. 2006). There are three steps to the ethical matrix method: 1) listing types of stakeholders in the rows of a values matrix, 2) expressing, in columns, each stakeholder’s interests concerning their well-being, autonomy (freedom), and justice, and 3) creating (through deliberation) a second, issue-specific consequences matrix focused on a decision (such as whether, when, or how to evacuate ahead of a hurricane’s landfall). The values matrix (online supplemental material S1; https://doi.org/10.1175/BAMS-D-22-0145.2) is used by participating stakeholders to understand each other’s views and values. The consequences matrix (supplemental material S2) aggregates stakeholders’ input and captures logistical challenges, trade-offs, and other concerns to encourage transparency and document joint decision-making.
To explore the effects of these experiences, we presented a scenario at a fictitious, coastally located college campus. We chose a fictitious campus to mitigate the potential influence of geographic distance on risk perception; our participants were drawn from colleges across the United States. This broad recruitment allowed us to explore how diverse participants responded to workshop components, while giving college students more appreciation for storm surge flood risk and preparedness issues, since they are the next generation of coastal residents in a changing climate with rising sea levels. The goal of this paper is to describe key workshop components, such as 1) comparing risk communication delivered through investigated impacts of standard weather forecasting graphics versus 3D flood-simulation visualizations, and 2) having participants role-play and discuss perspectives of multiple stakeholders considering evacuation. We also present participants’ comments concerning the workshop’s value. More comprehensive and quantitative results will be in a follow-up paper.
Workshop design
Participants.
Participants were recruited via university email and social media based on interests in climate change, decision-making, weather forecasting, emergency management, resilience, communication, and social justice. We recruited 91 undergraduates and 72 graduate students from around the United States (29 states) aged 18–64. Academic majors ranged widely, from communications and public health, to Earth and atmospheric sciences and other sciences, to social sciences and humanities. Eighteen people, including six members of the workshop planning team, served as facilitators for breakout groups. They received a facilitators’ guide and participated in at least one virtual training session prior to facilitation. Participants outside of the workshop team, who completed the 4-h workshop, received an $80 e-card.
Experience.
Participants began the workshop with a set of baseline survey questions about their experiences with extreme weather emergencies, as well as their attitudes and intentions concerning emergency preparedness for themselves and those living near them. Demographics were surveyed at the end of the workshop.
The workshop unfolded as a multipart narrative about a hurricane with the potential for devastating storm surge flooding. All participants experienced a prerecorded narrative about an approaching storm that was successively 5, 3, 2, and 1 day away from potential landfall on the coast of the fictitious college campus located only a few meters above sea level (Fig. 1). Included in the narrative was a depiction of characters (in student and staff roles typical for a college campus) and their thoughts, conversations, and any preparations as they anticipated the impacts of the storm. The narrative was punctuated with audiovisual weather forecasts from the campus television station (see supplemental material S3). At designated points during the narrative, participants completed survey questions about potential impacts and shared concerns affecting their decision-making (e.g., personal safety, academic concerns, such as delaying graduation) and their plans to evacuate. See Fig. 2 for the workshop timeline.
Campus layout near the coast looking top down with the buildings labeled. The red “x” denotes the location depicted in Fig. 3.
Citation: Bulletin of the American Meteorological Society 104, 7; 10.1175/BAMS-D-22-0145.1
Workshop timeline and tasks.
Citation: Bulletin of the American Meteorological Society 104, 7; 10.1175/BAMS-D-22-0145.1
For a randomized subset of 80 participants, television broadcasts were accompanied by 3D computer graphics visualizations (see Fig. 3). The other half of the participants were shown standard weather graphics as often seen in a television broadcast, with the same probabilistic forecast information as the 3D visualization group. As the storm approached, participants in the 3D visualization group saw simulations of flood water rising across the campus; these depicted both likely and worst-case scenarios, including flood waters rising, engulfing cars, and nearly submerging street signs.
Examples from the visualization animation show the flooding on campus for water levels of (left to right) 0, 3, 9, and 11.5 ft above sea level, respectively (1 ft ≈ 0.305 m).
Citation: Bulletin of the American Meteorological Society 104, 7; 10.1175/BAMS-D-22-0145.1
Visualization.
As noted, half of the workshop participants saw 3D renderings of flooding scenarios generated using a custom-built virtual reality application developed by our team (Fig. 3). This application allows the design of a 3D terrain with objects such as buildings, roads, paths, vegetation, light sources, and custom objects. Given an input flooding height, the application generates flood simulation visualizations that consider scene objects and terrain height and slope. The application was implemented using the OpenGL graphics library (Shreiner et al. 2009) and used graphics acceleration hardware to produce high-resolution graphics.
Ethical matrix.
After experiencing the first segment of the narrative and the first forecast, participants received a brief orientation to the ethical matrix method. Then, each was assigned to role-play one of six characters featured in the narrative. Characters included two students with different needs and backgrounds, a resident advisor in a dormitory, the university provost/head of student affairs, the director of building facilities, and a maintenance staff employee. In the first of two breakout sessions, six same-character groups of participants were introduced to their character’s backstory (Table 1). Then they worked together to fill out their character’s values matrix (concerning well-being, autonomy, and justice), considering issues such as lacking a safe place to evacuate to, or having young children at home.
Character summaries.
After the first breakout session, while the participants were experiencing the rest of the narrative leading up to the storm, a workshop facilitator united the six characters’ values matrices into a single consequences matrix with the different roles and perspectives visible in the same document. The remainder of the narrative included forecasts for two days before the storm, one day before the storm, and then the aftermath (with half of the participants continuing to experience 3D visualizations). Upon completing the survey questions, all participants were assigned to 1 of 10 mixed-character groups for their second breakout group session, where each participant role-played their assigned character and interacted with other participants role-playing the other characters. Each breakout group was moderated by one or more facilitators who encouraged discussion while populating the consequences matrix with the perspectives, needs, or concerns of each character. The groups discussed the practical and ethical trade-offs involved in preparing for a weather emergency and collectively weighed decisions that could maximize benefits and minimize harms.
Workshop feedback and outcomes
Participants’ feedback highlights the workshop’s strengths and limitations.
Participants.
Presenting the workshop online to over 160 participants had some benefits and drawbacks. A strength of the online workshop was that it allowed students from across the United States to attend. A problem of the online approach was participant focus and wait time. Because some participants did not have 3D visualizations, they moved more quickly through the workshop and often had to wait, while those who saw the 3D visualizations were more inclined to feel rushed. Another issue was unanticipated chatter in the main Zoom room, which was a distraction to those completing viewing and survey tasks. This feedback will be useful in orchestrating future workshops.
Experience.
Information alone rarely makes people change their minds, but personal experience often does (Slovic et al. 2004). The workshop’s design was informed by experiential learning theory, which takes an approach of “do, reflect, think, and apply” (Butler et al. 2019, p. 12). Students participated in scaffolded workshop experiences (do), considered these experiences in light of new information (reflect), developed their own understanding (think), and applied this to discussion and responses to survey questions (apply). A key component supporting the experience was the guiding narrative, where participants imagined they were associated with this coastal campus. Notably, research has found that narratives, in contrast to traditional textbook and lecture discourse, are easily processed and recalled. Moreover, people are less likely to argue with a narrative approach because they become involved with the plot and characters (Shaffer et al. 2018; Slater and Rouner 2002). This suggests that the experiential components of this workshop—its use of a narrative about a storm that could harm a university and visuals about the storm surge’s likely impact on familiar objects such as cars—would make hurricane-related flood threat a tangible, “feel-able” experience, encouraging reflection.
“What I found most valuable was giving everyone a specific character to learn about and role play as. I think this gave everyone the ability to learn that specific events that can occur, such as a weather disaster, impact everyones [sic] lives very differently. This gave a lot of insight to what people have to deal with.”
“I found the psychology of each end-user as to why they could or could not evacuate as easily to be fascinating. Oftentimes as scientists we can get caught up in the binary of directive advice to either be [sic] ‘evacuate’ or ‘not’ but realizing that it is not always so simple or that more resources could be directed to allowing others to follow the science, as well as recognize the magnitude of impacts, is helpful as a hopeful science communicator.”
Visualization.
Recall that half of the participants viewed 3D renderings of simulations showing flood waters rising near familiar objects, such as buildings and street signs, while the other half saw standard weather forecast graphics. Contrary to our expectations, those in the 3D graphical visualization group did not report that they would evacuate any earlier than those in the standard visualization group in the repeated-measures surveys staged 5, 3, 2, and 1 day before the storm (not shown). In fact, the evacuation response was 5% greater for the group who did not see the visualization, but this is not statistically significant. It is unclear whether this lack of a difference was due to some characteristic of the visualization, or to the effects of the narrative eclipsing any potential visualization effect, or to some other cause.
“I appreciate to see [sic] how the student newscast use simulation to communicate the threat of flooding. This is very helpful for viewers like me to understand the threat compared to just using words and maps.”
“The use of visual aids to convey important messages was the most interesting thing to note when it came to weather forecasting. The flood simulations really convey the magnitudes of the flood more than mere numbers and seeing the flood affect personal communities really portrayed a strong message.”
Ethical matrix.
“I found the emphasis on a difference in perspectives most valuable. Each individual in this workshop had their own view and each character we embodied had their own perspective and life history as well.”
“I found it most valuable to understand the gravity of such situations. And how not everyone is in the same position and needs different kinds of help and needs. This allows me to be more aware of myself and my actions to better help my community and those around me.”
Detailed quantitative analyses of these and other workshop outcomes (including from repeated measures in the online surveys mentioned in Fig. 2) will be reported in a longer forthcoming article. Preliminary analysis suggests that participants’ perspectives may have shifted as a result of the experience. For example, prior to working with the ethical matrix, participants’ comments about how the fictional storm would affect their lives focused on their own personal loss and experiences rather than on others. Afterward, participants expressed concern for others, including how others might be affected by the storm, along with an inclination to help others prepare. Comments also suggested that participants may have become more attentive to weather reports and storm preparations after the workshop.
Summary and next steps
In the face of a severe weather threat, improving communication with the public for better storm preparedness is vital. This workshop explored the value of visualization, experiential learning, and the ethical matrix for engaging college students with risks of storm surge, in order to measure changes in their attitudes and decision-making. A more quantitative assessment of the workshop survey will be completed to better understand the role of visualization versus human interactions on the decision to evacuate. We plan to complete a similar (hopefully in-person) workshop, with participants from coastal communities and community officials, first responders, etc., responsible for evacuations and safety. We hypothesize that the 3D visualization may be more useful if stakeholders have control over what they want to view, in order to explore additional information (evacuation routes, demographics, hospitals, etc.). We predict that this would be useful in real time to community officials in actual storm events. Facilitating interactions with real-world stakeholders using the ethical matrix could also help determine its utility for difficult decision-making and policy design for extreme weather events.
Acknowledgments.
We thank the three anonymous reviewers for comments that improved this manuscript. This material is based on work supported by NSF under Grant 1940302. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views and policies of the National Science Foundation and U.S. Environmental Protection Agency. Thanks to our volunteer facilitators and to SBU’s studio team for assistance in producing the weather forecasting videos. We acknowledge the Setalcott and Setauket Nations’ lands upon which we performed our research, and we pay our respects to their elders (past, present, and emerging).
Data availability statement.
Due to privacy and ethical concerns, no participant data are available at this time. Additional information regarding the values matrix, consequences matrix, and full character descriptions is available through the following link: http://bit.ly/3XkUsC7. An example 3D visualization video is available on YouTube: https://youtu.be/aapQjJa7gpk. The standard forecast video is available on YouTube: https://youtu.be/c_9yBMG6tsw.
References
Brust-Renck, P. G., V. F. Reyna, J. C. Corbin, C. E. Royerand, and R. B. Weldon, 2014: The role of numeracy in risk communication. The SAGE Handbook of Risk Communication, H. Cho, T. Reimer, and K. A. McComas, Eds., Sage, 166–192.
Burnside, R., D. S. Miller, and J. D. Rivera, 2007: The impact of information and risk perception on the hurricane evacuation decision-making of greater New Orleans residents. Sociol. Spectrum, 27, 727–740, https://doi.org/10.1080/02732170701534226.
Butler, M. G., K. S. Church, and A. W. Spencer, 2019: Do, reflect, think, apply: Experiential education in accounting. J. Accounting Educ., 48, 12–21, https://doi.org/10.1016/j.jaccedu.2019.05.001.
Dalwood, N., K. A. Bowles, C. Williams, P. Morgan, S. Pritchard, and F. Blackstock, 2020: Students as patients: A systematic review of peer simulation in health care professional education. Med. Educ., 54, 387–399, https://doi.org/10.1111/medu.14058.
Eva, P., and H. Ladislav, 2017: Virtual reality as needful factor of intervention in natural disasters. 2017 Int. Conf. on Engineering, Technology and Innovation, Hualien, Taiwan, IEEE, https://doi.org/10.1109/ICE.2017.8279861.
Forsberg, E. M., 2004: The ethical matrix: A tool for ethical assessments of biotechnology. Global Bioethics, 17, 167–172, https://doi.org/10.1080/11287462.2004.10800856.
Freeman, C. S., N. Nunnari, L. Edgemon, and K. Marsh, 2019: Improving public messaging for evacuation and shelter-in-place. Findings and recommendations for emergency managers from peer-reviewed research. FEMA Rep., 52 pp., www.fema.gov/sites/default/files/documents/fema_improving-public-messaging-for-evacuation-and-shelter-in-place_literature-review-report.pdf.
Fu, M., R. Liu, and Y. Zhang, 2021: Why do people make risky decisions during a fire evacuation? Study on the effect of smoke level, individual risk preference, and neighbor behavior. Saf. Sci., 140, 105245, https://doi.org/10.1016/j.ssci.2021.105245.
Lipp, N., N. Dużmańska-Misiarczyk, A. Strojny, and P. Strojny, 2021: Evoking emotions in virtual reality: Schema activation via a freeze-frame stimulus. Virtual Reality, 25, 279–292, https://doi.org/10.1007/s10055-020-00454-6.
Mepham, B., M. Kaiser, E. Thorstensen, S. Tomkins, and K. Millar, 2006: Ethical matrix manual. Agricultural Economics Research Institute (LEI) Doc., 45 pp., https://core.ac.uk/download/pdf/29269684.pdf.
Mol, J. M., W. J. W. Botzen, and J. Z. Blascch, 2022: After the virtual flood: Risk perceptions and flood preparedness after virtual reality risk communication. Judgment Decis. Making, 17, 189–214, https://doi.org/10.1017/S1930297500009074.
NOAA National Ocean Service, 2013: National coastal population report population trends from 1970 to 2020. Accessed 17 October 2022, https://coast.noaa.gov/digitalcoast/training/population-report.html.
Shaffer, V. A., E. S. Focella, A. Hathaway, L. D. Scherer, and B. J. Zikmund-Fisher, 2018: On the usefulness of narratives: An interdisciplinary review and theoretical model. Ann. Behav. Med., 52, 429–442, https://doi.org/10.1093/abm/kax008.
Shreiner, D., and Coauthors, 2009: OpenGL Programming Guide: The Official Guide to Learning OpenGL, Versions 3.0 and 3.1. Pearson, 936 pp.
Siebeneck, L. K., and T. J. Cova, 2012: Spatial and temporal variation in evacuee risk perception throughout the evacuation and return‐entry process. Risk Anal., 32, 1468–1480, https://doi.org/10.1111/j.1539-6924.2011.01781.x.
Simis, M. J., H. Madden, M. A. Cacciatore, and S. K. Yeo, 2016: The lure of rationality: Why does the deficit model persist in science communication? Public Understanding Sci., 25, 400–414, https://doi.org/10.1177/0963662516629749.
Simpson, M., L. Padilla, K. Keller, and A. Klippel, 2022: Immersive storm surge flooding: Scale and risk perception in virtual reality. J. Environ. Psychol., 80, 101764, https://doi.org/10.1016/j.jenvp.2022.101764.
Slater, M. D., and D. Rouner, 2002: Entertainment–education and elaboration likelihood: Understanding the processing of narrative persuasion. Commun. Theory, 12, 173–191, https://doi.org/10.1111/j.1468-2885.2002.tb00265.x.
Slovic, P., M. L. Finucane, E. Peters, and D. G. MacGregor, 2004: Risk as analysis and risk as feelings: Some thoughts about affect, reason, risk, and rationality. Risk Anal., 24, 311–322, https://doi.org/10.1111/j.0272-4332.2004.00433.x.
Toplak, M. E., R. F. West, and K. E. Stanovich, 2014: Assessing miserly information processing: An expansion of the cognitive reflection test. Thinking Reasoning, 20, 147–168, https://doi.org/10.1080/13546783.2013.844729.
Tversky, A., and D. Kahneman, 1992: Advances in prospect theory: Cumulative representation of uncertainty. J. Risk Uncertainty, 5, 297–323, https://doi.org/10.1007/BF00122574.
Uccellini, L. W., and J. E. Ten Hoeve, 2019: Evolving the National Weather Service to build a Weather-Ready Nation: Connecting observations, forecasts, and warnings to decision-makers through impact-based decision support services. Bull. Amer. Meteor. Soc., 100, 1923–1942, https://doi.org/10.1175/BAMS-D-18-0159.1.
Walsh, J. D., and Coauthors, 2014: Our changing climate. Climate change impacts in the United States: The Third National Climate Assessment, U.S. Global Change Research Program Rep., 19–67, https://doi.org/10.7930/J0KW5CXT.
