1. Introduction

Two institutions took contrasting approaches to using the questionnaire. At the University of Miami, teams of meteorology undergraduates evaluated the value chain for three hurricanes. Among the issues identified were the dynamic nature of the forecasts, misinterpretations of the products, social media influences, demographic factors, and disparities in responses. The Australian Bureau of Meteorology engaged student interns in different disciplines and experience levels to evaluate and contrast the warning value chains for domestic and international events.

The students expressed enthusiasm for the exercises. Educational benefits included team collaboration, critical thinking, research and composition skills, a comprehensive view of weather events, understanding information flow, learning about new tools, and identifying gaps in practices. We encourage educators to adopt similar exercises to enable students to develop these skills, adopt value chain ideas, and contribute meaningfully to the community. The level of maintenance is low, and there is flexibility in how the exercises can be developed.

2. What is a “warning value chain”?

Warning for weather disasters is complex. It is an iterative process that involves many connected outputs, processes, and disciplines, from the scientifically challenging preparation of timely and accurate predictions (and their uncertainties) through to the issuance of warnings and media to communicate them, and the understanding and preparation by officials and the public to minimize losses. As part of the World Meteorological Organization’s (WMO) High-Impact Weather Project (HIWeather), the economic concept of a “value chain” has been applied to explore and evaluate the flow of information in weather warnings (Fig. 1).

View Full Size

The value chain for high-impact weather warnings, with each of the six stages listed above the mountains. Adapted from Tan et al. (2022).

Citation: Bulletin of the American Meteorological Society 105, 8; 10.1175/BAMS-D-23-0273.1

• Download Figure
• Download figure as PowerPoint slide

The WMO value chain project team was formed in 2020 to explore value chain approaches and create an evaluative framework with guidance and tools. The project team, comprising over 40 members across many countries and professions, initiated a database of high-impact weather warning case studies. Using this database, scientists and practitioners can analyze previous experiences to better understand the warning value chain and identify potential improvements to mitigate future risk and damage. To provide a common framework for evaluation across many weather events, the project team developed a structured questionnaire in 2021 (see Ebert et al. 2024). The questionnaire spans the six stages in Fig. 1 from the use of observations through to decision-making, evaluating the outputs at each stage and the information exchanges between stages, culminating in a summary. To date, there are 18 case studies in the database (some of them are incomplete).

3. Students’ evaluations of the value chain

To add to the growing database and use the questionnaire as a guided educational tool for students interested in high-impact weather events, we developed contrasting approaches at the University of Miami (UM) and the Australian Bureau of Meteorology (the Bureau). UM used the same approach for three landfalling major hurricanes, and we highlight the students’ findings from these cases. The Bureau developed several approaches, described further below.

At UM, the first author wanted to add a dimension on impacts to his sophomore-level tropical weather and forecasting course. The value chain questionnaire provided an idea for a new assignment, using Hurricane Ida (2021) as a case study. To keep the assignment practicable and focused, we asked the students to select a subset of stages from Fig. 1. They preferred the hazard, impact, warning, and decision stages, comprising 47 questions. Example questions included the following: “How reliable and accurate were weather forecasts at different lead times?”; “Of those exposed, who and what were vulnerable to the hazards and why?”; and “To what extent were warning messages received and understood by the public?”

In 2021–23, we divided each class of 6–13 undergraduates into teams of ∼3 students, with each team assigned to one or two of the four stages. Each student was given an individual set of questions, to which they provided written responses in the form of a few paragraphs and accompanying graphics (Fig. 2), citing their sources. Each team then prepared a presentation that highlighted their findings, successes and failures, and challenges in doing the assignment. We encouraged the students to use online information from established agencies including NOAA and the Federal Emergency Management Agency (FEMA) together with conventional and social media and to use their judgment about the accuracy and representativeness of information. Aside from this guidance, the students were not supervised. The first author edited and compiled the students’ written responses into a large report and sent them to the second and third authors, who reviewed and further edited the report before uploading it to the database. WMO sent each student a certificate of appreciation.

View Full Size

Examples of graphics collected by the students to aid their responses for the Hurricane Ida (2021) value chain exercise.

Citation: Bulletin of the American Meteorological Society 105, 8; 10.1175/BAMS-D-23-0273.1

• Download Figure
• Download figure as PowerPoint slide

Hurricane Ida (2021) had multiple impacts, including its landfall as a category 4 hurricane in New Orleans, Louisiana, followed by its remnants flooding the northeastern United States, causing at least 45 deaths in the vicinity of New York. Findings from the hazard stage included the high streamflow that exacerbated the flooding and the benefits of a new FEMA impact forecasting tool. Impacts included oil shortages, toxic mold in and near Louisiana, and flooded basements in New York City which exposed the problem of illegal housing. The students identified an abrupt shift in communication from “A lot of uncertainty” to “Prepare now!” within 5 h and differences in urgency between “Category 4 Hurricane Ida” and “Remnants of Ida.” Across Ida and the other hurricanes, the widespread use of social media was identified, together with accessibility issues including language and mobility barriers. Some response efforts in Louisiana were compromised by the COVID-19 pandemic and extreme heat. While New Orleans prepared well, given memories of Hurricane Katrina (2005), examples of poor preparation in New York were identified, citing complacency and warning fatigue.

Hurricane Ian (2022), another category 4 hurricane at landfall, caused 149 deaths in Florida alone. The scale of the tragedy made this a somber exercise for the students. The teams noted that the hazard and impact forecasts were dependent on the complex track forecast and that some key forecast products were widely misinterpreted. The students identified positive and negative consequences of “disaster influencers” on social media. They also noted inconsistencies in evacuation protocols, with some officials taking an approach of caution (“just-in-case”) and others prioritizing efficiency and cost (“just-in-time”). More positively, the students commended the organizational response from FEMA, the preparedness and response plans in Florida law, and the availability of online resources such as preparedness programs, although the public uptake of these resources was questioned.

Hurricane Idalia (2023), a category 3 hurricane at its time of landfall in Florida, provided an opportunity to contrast the value chain evaluation to that of Ian. Using the 2022 report as context, the 2023 class provided more encouraging responses. The higher confidence in Idalia’s forecast led to lower uncertainty in the storm surge and wind forecasts, likely resulting in more consistent messaging which facilitated emergency preparation and decision-making. The forecasts were widely regarded as a success, and the recency of Ian may have bolstered the public response. Even though the storm surge was record breaking for that region, the damage was mitigated by Idalia striking a sparsely populated area. The students noted the high impact on the timber industry and that preparation and recovery were hampered in lower-income sectors. The multiple methods of public communication, together with strong dissemination and communication between agencies, were noted for all three hurricanes.

At the Bureau of Meteorology, the second and third authors advertised opportunities in 2022 and 2023 for student interns to work with them over a 3-month period, as a semester course for credit. In each year, five interns were accepted, comprising undergraduate honors and master’s students from several disciplines. Some interns worked on case studies, whereas others elected to help build the prototype database or develop an annotated bibliography on value chain approaches. Both groups of case study interns completed two events during their internship, working as a team for 3 h per week at the Bureau and doing the rest of their work remotely. In common with UM, the Bureau interns largely used online information and were encouraged to develop their own insights. They divided up sections of the questionnaire and collaborated on the questions, but in contrast to UM, their work was supervised.

The 2022 interns worked on the Australian Black Summer fires (2019–20) and the 2023 interns on the eastern Australian floods (2022). Their supervisors were able to provide them with Bureau data and reports that would have been difficult to obtain from outside a meteorological agency. The strong climate drivers in the lead-up to both events (El Niño and positive Indian Ocean dipole during the 2019–20 fires and strong La Niña during the 2022 floods) were understood by many Australians to mean dry and wet conditions, respectively. The most intense fires occurred during December 2019 and January 2020. Several hundred deaths were linked to heavy smoke concentrations, which was a wake-up call for the severity of risk to urban populations from wildfire. Despite fire being a prevalent hazard, a 2020 survey of more than a thousand young people found that 88% had no knowledge of how to defend themselves and the community in a state of disaster. In contrast, the many residents most affected by the eastern Australian floods had a flood action plan and experience of prior floods, though none so extreme as the one they experienced in 2022. Due to communication service outages, many experienced difficulties receiving relevant warnings during the event.

The strategy for the second case studies conducted by the Bureau interns was to compare warning value chains for the same type of hazard but for two different geographical regions with varying socioeconomic status. Evaluating the warning value chain in a developing and developed country gave insights to key differences in hazard preparedness, forecasts, impacts, warning communication, and response mechanisms. The 2022 students examined Hurricane Isaias (2020) as it passed through the Caribbean and United States, while the 2023 students compared the 2022 Australian floods with a heavy rain event in Durban, South Africa, in April 2022. The interns built on work initiated by the South African Weather Service to evaluate the weather forecasts and warnings for the Durban flood event, demonstrating how independent teams can contribute toward completing a full value chain evaluation. Each intern worked on a different value chain stage for their second case. Their familiarity with the template and how the warnings worked for their first case in Australia gave them a useful frame of reference. While the hazard forecast in the developing regions (South Africa and the Caribbean) was on par with that of the developed countries (Australia and the United States), due in part to the use of forecasts from international National Meteorological and Hydrological Services, the lower coping capacity of the population and barriers in warning communication, including language, led to more severe socioeconomic impacts in the less wealthy countries.

Another innovation at the Bureau was a single 3-h session doing a “flash case study” of an active hurricane. After dividing up the value chain, the interns went away and learned as much as they could in about an hour and then everyone shared what they found. The internships concluded with the students giving a joint Bureau seminar on their work.

4. Students’ feedback

The exercises were enthusiastically received by the students, who expressed gratitude for the opportunity to contribute to the WMO value chain project. Some of the educational benefits at UM were desired, whereas others were unexpected. These benefits included the enhancement of students’ research and composition skills; building a comprehensive view of weather events and their impacts, including from their classmates’ presentations about other stages; and retrospectively analyzing a storm with a known outcome, in contrast to only forecasting it. The students’ comments on what they had learned included rapid information changes and potential roadblocks to timely communication of lifesaving information; understanding the next steps in the chain after the forecast has been issued; new tools for hazards and warnings that contrast with the meteorological processes, observations, and models in the rest of the course; the differences in how information is provided to and used by emergency managers versus the public; what goes into disaster planning and recovery efforts; and identifying gaps in practices. One student pointed out that the project expanded her interest in future career paths outside weather forecasting. The consensus experience is reflected by one student, who stated that the process of working on the project was “interesting, beneficial, and fun.”

The UM students also cited the benefits of team collaboration, which helped build class spirit and was a refreshing change from the traditional classroom environment where students face the instructor and have limited interactions with each other. Other benefits included advancing students’ critical thinking; exposure to disciplines that are not traditionally in meteorology curricula (communication, psychology, law, economics, business operations, and public health); reflecting on how to improve elements of the warning value chain; resourcefulness in finding information; critically reviewing a large volume of information and distilling key findings, successes, and failures; and learning about uncertainty. The student interns at the Bureau, who were at the undergraduate honors and master’s levels, developed skills in understanding the complex nature of the high-impact weather warning process; using value chain concepts to understand and describe information flows; developing case studies; and interacting with professional scientists. The Bureau interns enjoyed doing the analysis part of the value chain as part of a group, including the subjective evaluation at the end. The “flash case study” was viewed as a fun experience.

5. Lessons learned

The value chain questionnaire facilitated the opportunity for students to contribute directly to WMO and the global community of researchers, forecasting agencies, stakeholders, and decision-makers at an early stage of their career. One reason to conduct these exercises during or shortly after a high-impact event is to capture many graphics and articles that quickly disappear. On the other hand, immediately available information is expected to be incomplete and inaccurate, and the resulting reports should not be viewed as comprehensive.

The exercises served the purpose of evaluating the questionnaire. Most questions elicited a detailed response, with a spectrum of answers in some cases. The archiving of products (e.g., at NOAA) facilitated informed responses about the evolution of the forecasts and official warnings, although formal evaluations require gridded data and rigorous statistical methods. There are additional limitations of our approach to case study compilation. While the impacts and responses are widely reported and provide material to curate a response, the information can be fragmented and stories may be anecdotal and not fully representative of a situation. The most difficult questions to address were those on impact forecasting (localized flooding, road closures, etc.) which remains in its infancy and internal communications and coordination between agencies. Additionally, quantifying impacts such as economic losses take months or years to complete. While some negative instances of poor communication may be identifiable, the positive impact of good communication is less straightforward to evaluate. The students also had to use their own judgment on the reliability of the information they retrieved.

Overall, we found the introduction of the value chain exercises as an educational tool to be rewarding, providing several unforeseen benefits to the students and the community. Given that the questionnaire was well structured and vetted, we found it straightforward to pass to the students, needing only limited additional guidance. The more experienced students were able to pursue the case study research in more depth. The students thrived on their elevated responsibility, and their enthusiasm and commitment to preparing their written responses and presentations were the main reason for its success. Based on the students’ feedback, our approaches generally worked to the first order.

6. Suggestions for educators

We encourage the incorporation of value chain exercises in meteorology curricula, for the mutual benefits of students and the broader community. In contrast to textbook-driven learning of more technical material, this exercise provides an opportunity for students to broaden their understanding of societal issues by taking the responsibility of being an evaluator. These types of studies appeal to the current generation of students, who have expressed a deep interest and concern about weather and climate threats. With high-impact events as the context, it offers students an opportunity to be introduced to important questions in disciplines that normally appear distant from meteorology.

There is flexibility in how the value chain exercises can be set up. The UM hurricane case assignments required little work by the instructor, given the well-formulated questionnaire. People at all levels, from high school through to experienced researchers, can contribute to the database. As the Bureau experience showed, the inclusion of students from a range of disciplines adds depth and breadth to the evaluations. The approach for students could vary from a homework assignment to an honors thesis to a graduate-level thesis and peer-reviewed publication. There is flexibility on which sections of the value chain to focus on, for example, a physical science study of the weather forecast, or a social and behavioral science study of the response, or the economics of planning and impacts. Value chains can be compared across different events, and multiple groups across the globe can contribute to a single case. The value chain can be extended to all types of high-impact weather events such as snowstorms or tornado outbreaks and to geophysical hazards such as volcanic eruptions and tsunamis. Finally, engagement with operational centers and stakeholders would deepen the assessments further.