Abstract

Weeklong weather science and safety workshops were conducted with 66 teachers of kindergarten through eighth grade (K–8) in three Georgia counties using the American Red Cross (ARC) Masters of Disaster (MoD) curriculum. The workshop goals included building teacher interests in the MoD, increasing teacher knowledge about the MoD curriculum, increasing and evaluating its use by teachers, disseminating information about it to other teachers, evaluating students’ weather science and safety knowledge, and evaluating students’ and families’ weather safety behavior. Workshop participation produced significant increases in teachers’ knowledge about the MoD curriculum, their general knowledge of weather science and safety, and self-efficacy in teaching their students about severe weather. In the year following the workshops, at least 32 teachers from the workshops delivered 178 MoD lessons to 2,465 students in K–8. In a sample of 291 students whose teachers delivered an MoD lesson on lightning, tornadoes, hurricanes, or floods, students obtained a mean of 60% correct responses on a comprehensive postlesson follow-up test. In a follow-up study with a subsample of 94 parents whose children received instruction from the MoD curriculum, 71% of the families indicated that they had developed safety plans and took additional steps (e.g., assembled safety kits, identified evacuation routes, and/or gathered supplies) to prepare for severe weather. This project is thought to be the first of its kind to demonstrate systematically the effectiveness of weather science and safety education for teachers, their students, and the students’ parents.

Weather safety workshops with teachers in Georgia conveyed weather safety lessons to thousands of students in kindergarten through eighth grade and made a difference in severe weather readiness at home.

Routine and extreme weather pose challenges to human health, safety, and daily functioning. National Weather Service statistics (www.nws.noaa.gov/om/hazstats.shtml) revealed that from 2006–15, an average of 110 Americans were killed annually by tornadoes, followed by 84 deaths annually due to flooding, 43 due to hurricanes, and 31 from lightning. The damage and losses from all weather hazards in the United States, including heat, cold, and wind as well as those mentioned above, averaged over $15 billion annually over this same period.

Such weather hazards underscore the importance of educating students, from kindergarten to grade 12 (K–12) about weather science and safety. A basic, systemic, and long-term effort is required to create a culture of weather safety among children and adults. The consensus from existing studies of responses to natural hazards suggests that providing education about the science, risks, and safety behaviors associated with such hazards is key in raising peoples’ awareness levels and in effecting adaptive changes in their responses when hazards threaten (Balluz et al. 2000; Blanchard-Boehm and Cook 2004; Brown et al. 2002; Liu et al. 1996; Stewart et al. 2015). In addition, dedicated instruction in weather science and safety is essential in creating a “Weather-Ready Nation,” which is a recent initiative of the National Weather Service to help communities and individuals prepare and respond effectively to severe and extreme weather (Lindell and Brooks 2013).

EXISTING WEATHER SCIENCE CURRICULUM RESOURCES.

An impressive collection of resources exist for K–12 educators to use in instructing students about weather science. The American Meteorological Society (AMS) developed Project Atmosphere to help teachers learn about the physical science of weather (Ginger et al. 1996). Project Atmosphere has a long and distinguished history in helping many teachers learn about weather. Similarly, the AMS DataStreme Atmosphere project aims to train weather education resource teachers so that they can assist teachers in local schools to incorporate further weather- and climate-related instruction for students (www.ametsoc.org/amsedu/). These AMS materials provide thorough and scientifically current lessons for instructing students about weather science.

Other resources and materials focus on observing various parts of the atmosphere. Two examples include the Global Learning and Observations to Benefit the Environment (GLOBE; www.globe.gov; Butler and MacGregor 2003) and the Students’ Cloud Observations Online (S’COOL; https://scool.larc.nasa.gov/; Chambers et al. 2003). These excellent materials focus more on the science of weather and climate in general, upon observing, and to some extent on forecasting. They are less focused, however, upon the specific topics of weather safety for lightning, tornadoes, hurricanes, and floods. Thus, the MoD curriculum is unique and additive because of its focus upon weather-related hazards and upon the themes of disaster preparation and recovery.

In this article, we describe our efforts to infuse weather science and safety concepts in four weather-vulnerable school districts in the state of Georgia, using the American Red Cross (ARC) Masters of Disaster (MoD) natural hazards science and safety curriculum, in three teacher workshops we conducted in 2011 and 2012. The workshops were designed to build teacher knowledge and commitment in using the MoD curriculum and in disseminating it to other teachers so that more students would benefit from the curriculum. Beyond training teachers, we examined the extent of student learning about weather hazards when teachers taught the MoD, and we surveyed a subgroup of the students’ parents to determine the effects of the school-based instruction on severe weather preparedness at home. To our knowledge, no other weather education effort has attempted to assess how students’ learning affected preparations for severe weather in the home. In conducting this project, our primary questions were as follows:

  1. Can teachers learn about weather science and safety in a professional development workshop using the MoD curriculum?

  2. Do the students whose teachers used the MoD curriculum show evidence of learning about weather science and safety?

  3. Is the use of the MoD curriculum in the classroom associated with greater knowledge and preparation for severe weather in the student’s home?

MASTERS OF DISASTER CURRICULUM.

ARC developed the first edition of the MoD curriculum in 1999, with input from state and federal agencies and private organizations [e.g., National Oceanic and Atmospheric Administration (NOAA)/National Weather Service, NOAA/National Severe Storms Laboratory, The Weather Channel, and U.S. Geological Survey], to provide a comprehensive curriculum for educating K–8 students on the science and safety of natural hazards. The MoD curriculum (see www.redcross.org/local/utah/programs-services/masters-of-disaster-program or contact the first author for a curriculum kit) consists of a series of ready-to-go lesson plans and teaching materials for grades K–2, 3–5, and 6–8. The major content of the MoD curriculum includes preparing for any disaster with “Be Disaster Safe”; coping with issues like terrorism, war, and pandemic flu with “Facing Fear”; recovering and rebuilding from any disaster with “In the Aftermath”; preventing injuries that happen at home with “Home Safety”; and keeping safe from fire with “Fire Prevention and Safety.” The MoD curriculum kit is approximately the size of a portfolio folder and includes the lessons, teaching guides, activity sheets, and other consumable materials on a CD. Lesson-related videos are contained on a DVD. Spanish-language materials have been included in the revision. The lessons are organized with a uniform format that consists of key terms and concepts, lesson purposes and objectives, activities, wrap-up, and linking across the curriculum.

Our teacher workshops provided extensive coverage of the four MoD weather science and safety modules—hurricanes, floods, lightning, and tornadoes. Each of these modules include an average of three to four lessons made up of activities that teachers could complete within approximately 20–45 min. In a typical module, approximately two lessons are dedicated to helping students learn about the science of the weather hazard, while the remaining lessons emphasize preparations and safe practices. The lessons within each module can be used separately or together as required by time or curricular guidelines. Importantly, each lesson plan has a “Home Connection” activity that gets safety and preparedness information into students’ homes and an assessment component that helps teachers assess students’ learning through activities and/or discussions. Ahead of conducting our workshops, we demonstrated the alignments of the MoD curriculum with the Georgia performance standards for science and also obtained a letter of support for our project from the Georgia Department of Education. Most recently, we have provided alignments of the MoD lessons with the Next Generation Science Standards (http://weathersafety.net/MoDNGSSAlignments.pdf).

WEATHER SCIENCE AND SAFETY WORKSHOPS.

Our goals in providing the weather science and safety workshops and in evaluating their impact with teachers, students, and parents were to 1) enhance teacher attitudes about using the MoD and build interest in the MoD curriculum, 2) increase teacher knowledge about the MoD curriculum, 3) increase and evaluate the usage of the MoD curriculum by teachers in the participating districts, 4) disseminate information about the MoD to teachers who did not participate in training, 5) evaluate changes in student weather science and safety knowledge, and 6) evaluate changes in student and family weather safety behavior. We evaluate our success in achieving each of these goals in the results section.

After establishing collaborative agreements with the participating school districts (Bibb, Calhoun, Glynn, and Brantley Counties) in Georgia, we circulated electronic and hard-copy descriptions of the workshop to recruit teachers for the workshops. Teachers made online applications. The workshop week consisted of 40–45 h of contact time at a central location within each school district (e.g., district offices, curriculum library). After we administered the preworkshop measures, we introduced the MoD curriculum features; described how it met the state performance standards for science in the K–8; and then provided presentations, demonstrations, and practice opportunities for the four primary MoD modules (lightning, tornadoes, hurricanes, and floods). Knox (second author) and a graduate student (Qureshi) delivered the weather science portion of the workshop training, while Stewart provided the instruction on the weather safety aspects of the MoD. The workshop included hour-long Skype tours that included a question-and-answer session with personnel from the Storm Prediction Center and the Weather Prediction Center. Finally, the teachers prepared and practiced delivering a lesson that they would use in the upcoming school year with their students; this occurred on the last day of the workshop. The workshop agenda of topics and activities appears in supplementary Table ES1 online (https://doi.org/10.1175/BAMS-D-17-0114.2).

The teachers received a stipend to attend the workshops and were provided with several MoD curriculum kits to use and to disseminate during the next school year. We offered three workshops in the summers of 2011 and 2012 to 66 teachers (58 women and 8 men) in Bibb, Calhoun, and Glynn Counties of Georgia. These teachers all worked in or near school districts that were located in climatologically and sociodemographically vulnerable areas with respect to the occurrence of severe or extreme weather according to an analysis conducted by Knox and Qureshi. Further details about the origin and development of our weather science and safety workshops using the MoD can be found in Stewart et al. (2015).

PROJECT MEASURES AND WORKSHOP EVALUATION.

Workshop measures.

We assessed our achievement of project goals both during and after the workshops. At the start of the workshop, the participants were asked to complete a preworkshop assessment packet, which included questions regarding their attitudes and beliefs about the teaching of the curriculum [the Weather Safety Education questionnaire (WSEQ)] and also a set of items intended to assess the teachers’ orientation to the workshop training goals. The participants completed a goal-attainment scaling procedure in which they articulated up to three of their own goals for the workshop and then rated the degree of importance for achieving each goal (Kiresuk et al. 1994). The next pair of measures assessed the teacher’s efficacy-related beliefs about providing instruction in science generally. Self-efficacy is an essential element in teachers both believing that they can provide instruction and that the instruction will lead to successful student learning outcomes (Bandura 1977, 1997). The teachers completed the Science Teaching Efficacy Belief Inventory (STEBI; 25 items; Riggs and Enochs 1990) and an adaptation of the STEBI that was specific to providing instruction in weather science and safety, Weather Science and Safety Teaching Efficacy Inventory (WSSTEI; 20 items) that was created by the first author. The teachers also completed a Weather Science and Safety Content Test (WSSCT) to assess their amount of available knowledge before beginning the workshop week. The WSSCT consisted of 22 multiple choice, true–false, and short-answer items. Approximately one-half of the measure was devoted to weather science, while the other half assessed content pertaining to weather safety practices. The workshop faculty developed an initial pool of items (nearly 50 in all) and then placed half of the items on the WSSCT as a pretest. We used the remaining half of the items as a posttest, which was a parallel form of the WSSCT pretest measure. We assessed the effects of the workshop in a pre- and posttest manner, with project measures administered on the first and last days of the workshop. Additional details on the development of these measures, their psychometric properties, and their use with teachers in earlier workshops are available in Stewart et al. (2015).

Postworkshop teaching and learning measures.

We also monitored teachers’ use of the curriculum and what students learned from it, as well as the parents’ awareness of the lessons during the subsequent school year. We observed a subsample of nine teachers (14% of all participants) deliver different MoD lessons to their classes. In addition, a subset of seven teachers administered postlesson content tests to 291 students. The teachers who volunteered for in-class observations and for the postlesson content tests were those who had been using the MoD lessons and who had the scheduling availability to accommodate the authors when we made the request. To assess student learning about weather science and safety, we developed 12 content tests: one for each of the MoD lesson areas of lightning, tornadoes, hurricanes, and floods and one for each of the three grade levels (K–2, 3–5, and 6–8). The content tests contained a mix of weather science and safety concepts operationalized in 10–19 objective and open-ended questions that were drawn directly from the MoD curriculum. The tests were designed to be general and comprehensive with respect to their coverage of MoD curriculum content.

Follow-up with teachers and parents.

Next, we followed up with the workshop teachers through an online Use and Dissemination questionnaire; 32 teachers (48%) responded to this instrument. Finally, we used our Home Connection survey with a subsample of 133 parents whose children had been taught from the MoD curriculum to assess their awareness and knowledge of weather science and safety for lightning, tornadoes, hurricanes, and floods. The design of the project and the use of all project measures were reviewed and approved by the University of Georgia Institutional Review Board. We report on the attainment of the project goals as revealed by the use of these measures below.

RESULTS.

Goal 1: Enhance teacher attitudes and build interest in the MoD curriculum.

The teachers in our workshops, via the Weather Safety Education questionnaire, estimated that a little over 25% of the students that they taught were adequately prepared to deal with hazardous weather. The teachers reported spending an average of about 15.5 h yr−1 providing instruction about general weather concepts. Of this time, slightly over one-third of the instructional time was devoted to topics pertinent to weather safety. Although 11 (16.7%) of the teachers believed it was important to instruct students about the weather hazards unique to their geographic area, a majority (n = 54, 81.8%) believed it was important to provide both instruction about weather hazards in general and also about those unique to their locale. None of the workshop participants had either known about or used the MoD curriculum previously. Admittedly, because teachers could self-select to attend the workshop, interest in the training and the MoD curriculum was high to begin with.

Goal 2: Increase teacher knowledge about the MoD weather science and safety curriculum.

Although the teachers brought a great deal of interest in weather to the workshop, they did not generally possess much knowledge about the science or safety of lightning, tornadoes, hurricanes, or floods. Their preworkshop mean score on the WSSCT was 8.00 out of 22 items [standard deviation (SD) = 2.62]. Their scores nearly doubled (mean M = 14.35, SD = 2.63) on a parallel form of this test that we gave at the end of the workshop week. The teachers answered significantly more items correctly at the end of the workshop (using a parallel form of the WSSCT) compared to their performance at the beginning of the workshop: F (1, 65) = 20.57, p < 0.0001. Although attending the workshop was associated with answering an additional six questions correctly, the mean posttest score (65% correct) indicated that there was still additional science and safety content that the teachers had not fully understood or mastered. A content analysis of the teachers’ posttest responses revealed that the knowledge about tornado science and safety was significantly greater (M = 75% correct; 95% confidence interval: 71%–80%) than their knowledge in the areas of lightning, floods, and hurricanes (with means ranging from 62% to 65% correct). These results were consistent with our observations during the workshops that the teachers showed much interest in tornado science and safety. Further, one of the counties (Bibb) had recently been affected by tornadoes, which may have built extra interest in the topic.

Goal 3: Increase and evaluate the usage of the MoD curriculum by teachers in the participating districts.

Teaching efficacy beliefs.

We evaluated both the workshop teachers’ beliefs and behaviors in assessing this goal. First, teaching efficacy beliefs are important in helping teachers to feel competent to provide successful instruction to their students. In this regard, we observed that the teachers increased their overall mean scores on the STEBI by over four points: 92.08 (SD = 11.26) at pretest to 96.42 (SD = 11.71) at posttest. The maximum possible score on the STEBI is 125. A repeated measures analysis of variance revealed that the score increase in the STEBI was statistically significant: F (1, 65) = 16.74, p < 0.0001. The WSSTEI provided a more specific assessment of the levels of efficacy beliefs that the participants possessed about weather science and safety. The teachers’ scores on this measure were as follows: pretest M = 69.82 (SD = 10.75); posttest M = 75.91 (SD = 6.20). Overall, attending the workshop was associated with a nearly six-point increase in the total scores on the WSSTEI: F (1, 65) = 20.22, p < 0.0001. Teachers’ beliefs that they could successfully teach weather science and safety topics increased during the workshop.

At the conclusion of the workshop, we also asked teachers to use a five-point rating scale (from 1 = strongly disagree to 5 = strongly agree) to respond to the following global evaluative statements: “Overall, I believe that I can use the Master of Disaster curriculum to teach my students how to behave safely when severe weather threatens them” and “Overall, students can learn from the Master of Disaster curriculum how to behave safely when severe weather threatens them.” The respective postworkshop means and standard deviations for these questions were M = 4.69 and SD = 0.68 and M = 4.65 and SD = 0.69, respectively. In following up two years after the workshop with 26 teachers (39% of the original cohort) who responded, we observed that teachers’ beliefs in the MoD curriculum were stable and that teaching self-efficacy beliefs were maintained; there were no statistically significant differences across the 2-yr time span. That is, teachers believed that they could use MoD to teach their students how to respond to severe weather (M = 4.20, SD = 1.04) and that their students could learn from the curriculum (M = 4.60, SD = 0.87).

Use of the curriculum.

Regarding the actual use of the MoD curriculum during the following school year, 32 teachers delivered 178 MoD lessons to a total of 2,465 students in K–8. In Table 1, we summarize the number and type of MoD lessons that teachers used with their students along with their degree of usage of the MoD curriculum. Overall, the teachers reported using the MoD curriculum “somewhat” to “much” in educating their students about weather science and safety. Table 2 shows that with the exception of the disaster preparation (Be Disaster Safe; M = 4.40) lesson for grade 6, teachers’ reported that their use of the MoD generally does not rise above a rating of 4.0 (corresponding to much use).

Table 1.

Amount of MoD lesson use by grade K–2, 3–5, and 6–8 teachers who responded to the Dissemination and Use questionnaire. The survey respondents indicated their extent of MoD lesson use via a five-point rating scale: 1 = not at all, 2 = a little, 3 = somewhat, 4 = much, and 5 = completely. Values with an asterisk represent the mean of means for the extent of MoD lesson use. Values in parentheses are percentages of the total numbers.

Amount of MoD lesson use by grade K–2, 3–5, and 6–8 teachers who responded to the Dissemination and Use questionnaire. The survey respondents indicated their extent of MoD lesson use via a five-point rating scale: 1 = not at all, 2 = a little, 3 = somewhat, 4 = much, and 5 = completely. Values with an asterisk represent the mean of means for the extent of MoD lesson use. Values in parentheses are percentages of the total numbers.
Amount of MoD lesson use by grade K–2, 3–5, and 6–8 teachers who responded to the Dissemination and Use questionnaire. The survey respondents indicated their extent of MoD lesson use via a five-point rating scale: 1 = not at all, 2 = a little, 3 = somewhat, 4 = much, and 5 = completely. Values with an asterisk represent the mean of means for the extent of MoD lesson use. Values in parentheses are percentages of the total numbers.
Table 2.

Summary of students’ performances on postlesson MoD content tests.

Summary of students’ performances on postlesson MoD content tests.
Summary of students’ performances on postlesson MoD content tests.

Teacher observations.

Nine teachers from the Bibb and Glynn County workshops gave their consent for us to observe their delivery of an MoD lesson. The teachers informed us prior to the visit about the material that they intended to cover; we had reviewed the lessons and were prepared to observe the teachers’ delivery of the material. The MoD lessons are designed to be flexible and adaptable to the different grade and intellectual levels of the students. Overall, the teachers all adhered well to the structure, format, and activities of MoD lessons that they chose to deliver. If the teachers did change or alter the lessons that they delivered, the modifications typically took the form of additions or supplements to the basic MoD lesson framework (rather than as deletions and substitutions) that were designed to respond to the unique needs of their students. Thus, the teachers’ fidelity to the MoD lessons was impressive.

Of the lessons that the teachers delivered during the visit, “The Hydrologic Cycle” was a popular choice. The lesson conveys basic science and also contains hands-on learning activities that illustrate the power of water and the necessity of behaving safely during extreme rain/flood events. A sixth-grade teacher in Bibb County, for example, fully implemented The Hydrologic Cycle, which included an activity where students used large aluminum baking sheets on which to construct a landscape that then received heavy rain (via the watering sprinkler) to simulate erosion and flooding. A first-grade teacher in Bibb County delivered a lesson on lightning to her first-grade class. In addition to emphasizing staying safe during lightning, the teacher also included a supplemental reading (listed in the MoD bibliography and titles that we had on display during our workshop), Polacco’s (1990)  Thunder Cake. This book helps children to understand thunder and also helps them to be less fearful of the loud thunder from a thunderstorm. We visited the four teachers in Brantley County during the same day. They had worked as a team to deliver the lessons and allow classroom observation at the same time. The teachers each emphasized a different lesson during the visit and planned to “rotate” the lessons so that their third-grade students each received coverage of lightning, tornadoes, hurricanes, and floods. Overall, the teachers used the MoD curriculum well and adapted it seamlessly with their existing curriculum materials.

Goal 4: Disseminate the MoD curriculum to teachers who did not participate in training.

The 32 teachers who responded to the Use and Dissemination questionnaire indicated that they discussed the curriculum with a total of 203 fellow teachers. The number of teachers with whom the workshop participants discussed the curriculum ranged from 2 to 20 teachers: M = 6.55, SD = 4.22. There were 25 (78.1% of those who responded) teachers who gave all of their extra MoD kits to other teachers. Two teachers indicated that they could use additional kits to give to other teachers. We asked the teachers to indicate the frequency with which they discussed MoD with their colleagues. The outbreak of severe weather at or near the teachers’ schools represented the occasion where teachers most often reported discussing the curriculum (n = 20, 62.5%). Learning about severe weather outbreaks that occurred somewhere else (beyond their locale) was the second most frequent occasion for discussing the MoD curriculum (n = 16, 50.0%). A weather safety or awareness week (e.g., tornado safety awareness week) was the third most frequent occasion for discussing the MoD (n = 13, 40.6%). The teachers reported that their colleagues taught the MoD to an estimated 3,096 students (M = 123.8 students, median = 102 students).

A final item in this section of the questionnaire asked the teachers about the likelihood of recommending the Weather Science and Safety workshop to a teacher colleague if it were to be offered in the future. The teachers used a five-point rating scale (from 1 = very likely to recommend to 5 = very unlikely to recommend). Overall, the teachers were favorably disposed to recommending the workshop to their colleagues: M = 1.47, SD = 0.56. The mean value fell between the “very likely to recommend” and “likely to recommend” anchors of the rating scale.

Goal 5: Evaluate students’ weather science and safety knowledge.

Seven of the 66 workshop teachers agreed to administer the postlesson tests to their classes following the coverage of one or more MoD lessons for their grade level. Five of the seven teachers were the same ones who allowed us to observe them during the MoD lesson delivery. The four MoD curriculum areas of lightning, tornadoes, hurricanes, and floods were all covered although generally by differing grade levels. That is, all three grade categories (K–2, 3–5, and 6–8) did not take the content test from the four curricular areas because we did not have a sufficient number of teachers from the different grades to deliver lessons from the four areas. We provide summary information about the lessons, class sizes, and the outcomes of the postlesson tests in Table 2.

Overall, on the 291 postlesson assessments that students completed, the mean proportion of correct responses was approximately 60%. Although this percentage ostensibly appears low in magnitude, several considerations can put the students’ achievement on the postlesson tests into perspective. First, teachers often taught one science and one safety lesson from the respective content areas (lightning, tornadoes, hurricanes, and floods). As we indicated previously, however, we designed the tests to be general and comprehensive. Therefore, when students completed the postlesson measure, they may not have received some of the instruction that would have been necessary to provide correct responses to the measure.

We examined the students’ performances on the postlesson tests for lightning and tornadoes, given that the majority of teachers delivered instruction on these MoD topics. In the K–2 test for lightning, the students most frequently missed an item covered in the first lesson that involved separating electrical charge by shuffling their shoes over carpet and then discharging the accumulated charge by touching an object (69% correct responses). The second item that K–2 students missed frequently pertained to lightning safety (also 69% correct). This item involved making an estimate of the distance of lightning based upon an elapsed amount of time between the lightning flash and the thunder. This content was associated with the second MoD K–2 lightning lesson. In the grades 3–5 postlesson lightning test, the students most frequently missed a question about how a thunderstorm cloud is like a capacitor (i.e., it stores electrical charge) that appeared in the first lesson (36% correct). They also frequently missed (45% correct) all or parts of a second-lesson question that asked them to label a diagram to accurately indicate 1) typical locations of positive charges and 2) locations of negative charges, a stepped leader, a return stroke, and cloud-to-ground lightning. Qualitatively, students generally were accurate in locating negative charges at the base of the cloud and positive charges at the ground; they made more errors in identifying the stepped leader. Finally, the students frequently missed a safety-related item from the fourth lesson that asked (true or false) whether it was safe to touch a person who had been struck by lightning (27% correct).

With respect to the tornado postlesson test, students regularly missed a first-lesson item that asked them to name the rotation inside of a thunderstorm that can lead to a tornado (11% correct). Rather than selecting the correct answer (i.e., mesocyclone), students tended to respond with “funnel cloud” or “tornado.” The students also frequently missed a question from the second MoD tornado lesson that involved the sources of moisture that can contribute to favorable conditions for tornadoes (17% correct). They were asked to indicate why the Atlantic Ocean was not a good source for moisture in the Midwest “Tornado Alley.” The remaining item, also from the second lesson, asked the students to name the collision of advancing cold air with a warm air mass (i.e., a cold front, 16% correct).

Beyond these item-level patterns, the students’ responses generally revealed that they understood and were able to reflect some of the principal safety facts related to the type of weather lesson that they received. Consequently, we were encouraged that the students had achieved some degree of learning and that the specific safety content embodied in the MoD curriculum was reflected in their postlesson test performances. Although this weather safety specificity part of the postlesson tests supported that students had learned from what their teachers delivered from the MoD curriculum, it was not possible to determine further how much of the students’ weather science–related performances was due to their exposure to the MoD curriculum and what they might have learned from other sources or in earlier grades.

Goal 6: Evaluate the weather safety behavior of students and families.

We evaluated this goal from the perspective of the students’ teachers and from a survey of parents whose children were exposed to the MoD curriculum. Regarding the teachers’ perspectives, the final section of the Use and Dissemination questionnaire asked the teachers to provide an estimate of the extent to which they believed that their students’ parents were aware of the lessons that they taught from the MoD curriculum. The teachers provided their estimates using a four-point rating scale (from 1 = not at all to 4 = always). For teachers in the grades K–2 and 3–5 categories, the responses ranged from 1 to 3. The mean for K–2 was between the “occasionally” and “frequently” anchors (M = 2.36, SD = 0.50). We observed similar results for students in grades 3–5 (M = 2.20, SD = 0.79), which again was between occasionally and frequently with respect to parental awareness. Teachers in grades 6–8 produced a mean estimate of 2.00 (occasionally; SD = 0.50).

One of the primary goals of this project involved making the parents of the students who were exposed to the MoD curriculum at school more aware of weather safety issues and practices. The presumed mechanism for reaching the parents with the MoD curriculum involved two related features: 1) the teachers’ use of the MoD curriculum with their students and, more specifically, 2) using the Home Connection feature that was incorporated in many of the MoD weather science and safety lessons. The Home Connection aspect of MoD typically involved the completion of homework assignments that involved the student’s parents and family. To assess the effects of such homework activities and the parents’ involvement, we developed the Home Connection questionnaire. The items of this questionnaire appear in Table 3. The instrument contained seven yes–no questions. If the parents responded “yes” to the last item (“As a result of my child’s learning about weather safety at school, we have taken steps at home to prepare for severe weather—like tornadoes, lightning, hurricanes, and floods”), we asked them to briefly describe the steps that they or the family had taken at home to prepare for severe weather. The teachers sent the questionnaires home with their students to the parents who would then complete them and return them via the U.S. Postal Service. As an inducement to increase participation, we paid parents $10 for completing and returning the form. It was difficult to control the time interval between the teacher’s MoD lesson delivery and the completion of the form by the students’ parents at home. In general, we estimate that the parents typically completed the forms within one month of their students’ involvement in a weather science or safety lesson. Based upon the teacher’s class size, we estimate that of the Home Connection questionnaires sent home, approximately 35% of them were returned.

Table 3.

Parents’ responses to the Home Connection questionnaire (n = 133). Values in parentheses are percentages of the number total.

Parents’ responses to the Home Connection questionnaire (n = 133). Values in parentheses are percentages of the number total.
Parents’ responses to the Home Connection questionnaire (n = 133). Values in parentheses are percentages of the number total.

The responses to the Home Connection questionnaire are summarized in Table 3. The data were encouraging in that, across the seven questions that we asked, a wide preponderance of the responses indicated that the parents had some degree of contact with the child’s learning experiences concerning weather science and safety. Approximately 70% of the parents indicated that they had known of or had been involved with some aspect of their child’s learning about weather science and safety. It is not likely that the parents were responding to non-MoD-related schoolwork or homework because we designed the Home Connection questionnaire to be especially responsive to the curriculum features that the students and parents might experience, especially as this involved safety (items 2, 4, 6, and 7). Because the MoD curriculum is both unique and specific in emphasizing safety around tornadoes, hurricanes, lightning, and floods, it is unlikely that the parents’ responses in Table 3 reflected non-MoD curriculum coverage. In addition, the teachers who distributed the Home Connection questionnaire responded to our Use and Dissemination survey as having relied principally upon the MoD curriculum with their students.

There were 94 parents who indicated that as a result of their child’s learning about weather safety at school, steps had been taken at home to prepare for severe weather (item 7). This represented nearly 71% of the parents who responded to the survey. Ninety-three parents provided comments to explain the nature of their family’s preparations. The parents’ comments ranged from statements that indicated a general readiness for severe weather—“We have gone thru examples of what to do and where to go during severe weather,” “We have emergency kits prepared, also a generator for power plus evacuation information on hand at all times”—to more specific plans for particular kinds of weather hazards: “We will go to our downstairs bathroom if there is a tornado. Stay indoors if there is lightning,” “We have a household plan for tornadoes like meet in the bathroom. Stay inside for lightning. Go to safe place in case of hurricane (out of town).” The parents most frequently mentioned their preparations for tornadoes (27 comments), followed by lightning (13 comments). Some parents mentioned severe weather or storms in general (15 comments each). Finally, there were 14 comments that mentioned safety or a safe place to shelter during severe weather. Overall, the parents’ comments were encouraging and conveyed a degree of readiness for severe weather.

DISCUSSION AND CONCLUSIONS.

Limitations.

We begin our discussion of the results by noting some of the project’s limitations. The first limitation involved the sizes of our samples as the project progressed in time. From our prior experience in conducting the workshops from 2008 to 2010, we observed that not all of the teachers who attended the workshop chose or were able to implement the MoD curriculum during the subsequent school year (Stewart et al. 2015). Such was also the case in the present project despite our being more stringent in emphasizing, both during the workshop application and training phases, the importance of actually delivering MoD lessons when possible. Between school years, teachers may move to different grades or to different schools, and this can affect their ability to implement the curriculum. This effectively limited the numbers of teachers who were subsequently taught the lessons, who allowed us to observe their instruction, who administered postlesson content tests, and who volunteered to send the Home Connection questionnaire home to their students’ parents. Given the longitudinal nature of this project, some degree of attrition is to be expected (Shadish et al. 2002).

Another and related limitation involves selection bias. We deliberately recruited teachers who expressed an interest in weather science and safety and a desire to learn how to deliver the MoD curriculum to their students. Our workshops were limited further by selecting teachers from within the state of Georgia. From our Weather Safety Education questionnaire, many of our teachers reported being dissatisfied with current curriculum resources for weather science and safety and routinely conveyed anecdotal experiences of severe weather affecting their locations in the past. These previous experiences made the weather salient for many teachers and were responsible, in part, for their interest in our workshop (Stewart 2009). Our results thus generalize only to teachers who have had similar kinds of weather-related experiences as teachers and who possess an above-average level of interest to provide instruction about weather science and safety.

A third limitation of our study is that we did not utilize any control or comparison groups for teachers, their students, or the students’ parents. Because we recruited teachers from Georgia counties that were socioeconomically vulnerable to the effects of severe and extreme weather and because many of these counties possessed a large number of students from underserved groups, we enrolled virtually all of the interested teachers in the workshops. That is, we did not have a sufficiently large pool of interested and willing teachers for some of them to serve as a no-workshop control/comparison group. Similarly, it was difficult to recruit teachers who were not in the workshop to administer postlesson content tests to their students. Anecdotally, once teachers in a grade had learned about the MoD curriculum from a teacher who participated in the workshop, they were interested in using the curriculum. They were somewhat self-conscious, however, about allowing us to assess their students out of concerns that their instruction may not be on par with the teachers who did attend the workshop. These are all significant limitations of our workshop evaluation. Nonetheless, the results seem sufficiently compelling such that our future efforts in providing teacher training can be designed (and funded) so that control and comparison groups can be used.

Weather science and safety education was effective.

This project extends our earlier work that demonstrated that Georgia teachers could increase their knowledge of weather science and safety through our workshops and also disseminate the results of their professional learning to both their teaching colleagues and to their students (Stewart et al. 2015). In the present project, not only did the teachers build their weather-related knowledge and sense of teaching self-efficacy, they also were able to convey this instruction to their students. Further, by involving their parents in the MoD-related Home Connection activities, it appears that parents became both more knowledgeable and better prepared for severe and extreme weather. To our knowledge, this project is the first of its kind to demonstrate systematically the effectiveness of weather science and safety education for teachers, their students, and the students’ parents. That is, our workshops were effective in the sense that by training and equipping teachers with the MoD curriculum, we were able to reach students and their parents in a way that made them more aware of and perhaps better prepared for severe and extreme weather.

Considerations in working with teachers and school districts.

Based upon our experiences in this project, several considerations are salient in moving forward to diffuse the MoD curriculum to educators in a broader manner. First, although ARC worked at the national level with agencies such as the Storm Prediction Center, the National Hurricane Center, and the National Weather Service to develop a broad curriculum that encompassed most of the weather hazards that people might experience, teachers at the local level must work within the parameters of their local school districts to use resources like the MoD. The systematic inclusion of the Masters of Disaster or other weather or climate curricula must align with the educational performance standards in the state where the teaching occurs—otherwise, teachers cannot use it in more than a minor way. In Georgia, we had to demonstrate how the various MoD lessons satisfied Georgia performance standards for science at each grade level so that teachers could use the curriculum along with their existing instructional materials and texts. Showing teachers how the MoD and other weather-related curriculum materials satisfy performance standards will greatly facilitate its regular use in classroom instruction. In this regard, we have aligned the MoD with the Next Generation Science Standards, which, as of 2016, had been adopted by 18 states and the District of Columbia (Heitin 2016).

Second, we have learned that teachers derive much benefit from professional learning workshops at both the content and process levels. In our workshop, we helped teachers become more familiar with the science and safety content associated with various forms of severe weather. Beyond this, we helped them with the processes of thinking about and practicing the implementation of the curriculum within their schools. Providing this kind of professional development is important because although teachers may possess much interest in weather, they may not have had much science education training within their degree programs. For example, the prototypical fourth-grade teacher in our workshop earned a degree in elementary education and had completed one or two science education courses. Although this was sufficient for work as a teacher in a self-contained classroom, many teachers viewed weather as a specialty science that required them to have more detailed knowledge of the physical and Earth sciences before they felt confident in using the MoD. We conclude from this that teachers would benefit from greater exposure to the atmospheric and Earth sciences in their preparation programs and that current teachers would continue to benefit from professional development such as we provided in our workshops.

The MoD curriculum complements existing resources.

The Masters of Disaster curriculum joins an impressive collection of resources for K–12 educators that exist in Project Atmosphere (Ginger et al. 1996), DataStreme, GLOBE (Butler and MacGregor 2003), and S’COOL (Chambers et al. 2003), among others. Although these resources provide excellent coverage of weather science, they are less focused upon the specific topics of weather safety for lightning, tornadoes, hurricanes, and floods. Thus, the MoD curriculum can make a unique contribution because of its focus upon weather-related hazards and upon the themes of disaster preparation and recovery. Along with the existing AMS, NASA, and National Weather Service online resources (www.weather.gov/owlie/ and www.weather.gov/safety), the use of the MoD curriculum can help achieve a “Weather-Ready Nation” (National Weather Service 2013). The authors hope that using the MoD curriculum will enable children and their families to better prepare and respond to severe weather.

ACKNOWLEDGMENTS

This material is based upon work supported by the National Science Foundation (NSF) under Grant 1034853 to the authors Stewart and Knox. The authors also acknowledge the assistance of Azadeh Fatemi, Aneela Qureshi, and Amy Todey in conducting the workshops. The authors thank the anonymous reviewers for their helpful suggestions.

REFERENCES

REFERENCES
Balluz
,
L.
,
L.
Schieve
,
T.
Holmes
,
S.
Kiezak
, and
J.
Malilay
,
2000
:
Predictors for people’s response to a tornado warning: Arkansas, 1 March 1997
.
Disasters
,
24
,
71
77
, https://doi.org/10.1111/1467-7717.00132.
Bandura
,
A
.,
1977
:
Self-efficacy: Toward a unifying theory of behavioral change
.
Psychol. Rev.
,
84
,
191
215
, https://doi.org/10.1037/0033-295X.84.2.191.
Bandura
,
A
.,
1997
: Self-Efficacy: The Exercise of Control. W. H. Freeman, 604 pp.
Blanchard-Boehm
,
R. D.
, and
M. J.
Cook
,
2004
:
Risk communication and public education in Edmonton, Alberta, Canada on the 10th anniversary of the “Black Friday” tornado
.
Int. Res. Geogr. Environ. Educ.
,
13
,
38
53
, https://doi.org/10.1080/10382040408668791.
Brown
,
S.
,
P.
Archer
,
E.
Kruger
, and
S.
Mallonee
,
2002
:
Tornado-related deaths and injuries in Oklahoma due to the 3 May 1999 tornadoes
.
Wea. Forecasting
,
17
,
343
353
, https://doi.org/10.1175/1520-0434(2002)017<0343:TRDAII>2.0.CO;2.
Butler
,
D. M.
, and
I. D.
MacGregor
,
2003
:
GLOBE: Science and education
.
J. Geosci. Educ.
,
51
,
9
20
, https://doi.org/10.5408/1089-9995-51.1.9.
Chambers
,
L. H.
,
D. F.
Young
,
P. K.
Costulis
,
P. T.
Detweiler
,
J. D.
Fischer
,
R.
Sepulveda
,
D. B.
Stoddard
, and
A.
Falcone
,
2003
:
The CERES S’COOL project
.
Bull. Amer. Meteor. Soc.
,
84
,
759
765
, https://doi.org/10.1175/BAMS-84-6-759.
Ginger
,
K. M.
,
J. M.
Moran
,
R. S.
Weinbeck
,
I. W.
Geer
,
J. T.
Snow
, and
D. R.
Smith
,
1996
:
Project Atmosphere: 1995 teacher enhancement programs
.
Bull. Amer. Meteor. Soc.
,
77
,
763
769
, https://doi.org/10.1175/1520-0477(1996)077<0763:EA>2.0.CO;2.
Heitin
,
L.
,
2016
: Hawaii adopts the Next Generation Science Standards. Education Week, accessed 21 May 2017, http://blogs.edweek.org/edweek/curriculum/2016/02/hawaii_adopts_the_next_generation_science_standards.html?cmp=SOC-SHR-TW.
Kiresuk
,
A. Smith
, and
J. E.
Cardillo
, Eds.,
1994
: Goal Attainment Scaling: Application, Theory, and Measurement. Lawrence Erlbaum, 326 pp.
Lindell
,
M. K.
, and
H.
Brooks
,
2013
:
Workshop on Weather Ready Nation: Science imperatives for severe thunderstorm research
.
Bull. Amer. Meteor. Soc.
,
94
,
ES171
ES174
, https://doi.org/10.1175/BAMS-D-12-on00238.1.
Liu
,
S.
,
L. E.
Quenemoen
,
J.
Malilay
,
E.
Noji
,
T.
Sinks
, and
J.
Mendlein
,
1996
:
Assessment of a severe-weather warning system and disaster preparedness, Calhoun County, Alabama, 1994
.
Amer. J. Public Health
,
86
,
87
89
, https://doi.org/10.2105/AJPH.86.1.87.
National Weather Service
,
2013
: Weather-Ready Nation roadmap. National Weather Service Rep., 75 pp., www.nws.noaa.gov/com/weatherreadynation/files/nws_wrn_roadmap_final_april17.pdf.
Polacco
,
P.
,
1990
: Thunder Cake. Putnam and Grossett, 32 pp.
Riggs
,
I.
, and
E.
Enochs
,
1990
:
Towards the development of an elementary teacher’s science teaching efficacy belief instrument
.
Sci. Educ.
,
74
,
625
637
, https://doi.org/10.1002/sce.3730740605.
Shadish
,
W. R.
,
T. D.
Cook
, and
D. T.
Campbell
,
2002
: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin, 623 pp.
Stewart
,
A. E.
,
2009
:
Mind the weather
.
Bull. Amer. Meteor. Soc.
,
90
,
1833
1841
, https://doi.org/10.1175/2009BAMS2794.1.
Stewart
,
A. E.
,
J. A.
Knox
, and
P.
Schneider
,
2015
:
Piloting and evaluating a workshop to teach Georgia teachers about weather science and safety
.
J. Geosci. Educ.
,
63
,
271
284
, https://doi.org/10.5408/14-069.1.

Footnotes

A supplement to this article is available online (10.1175/BAMS-D-17-0114.2).

© 2018 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Supplemental Material