A summer course has been developed at Purdue University that leverages students’ intrinsic desire to observe tornadoes as a motivator for learning severe storms forecasting. Relative to previous “storm chasing” courses described in the literature, the Students of Purdue Observing Tornadic Thunderstorms for Research (SPOTTR) course is enhanced by active learning exercises, career exploration activities, and the inclusion of research-grade meteorological instrumentation in order to provide an authentic in-field experiential learning scenario. After teaching severe weather forecasting skills and deployment techniques for several meteorological instruments (such as a mobile radar, radiosondes, and disdrometers), the instructors then guide the students on a 1-week field trip to the Great Plains, where the group executes a miniature field campaign to collect high-quality meteorological observations in and near severe storms. On days with no targetable severe weather, the participants visit sites deemed beneficial to the students’ professional development. The final week of the course is spent performing retrospective case studies based on the observations collected, and distilling lessons learned. Surveys given to SPOTTR students show that students’ understanding of severe storms forecasting, technical skills, and career aspirations all improved as a result of having participated in the SPOTTR course, affirming the efficacy of the course design.
At Purdue University, atmospheric science undergraduate and graduate students in the Earth, Atmospheric, and Planetary Sciences (EAPS) department have consistently expressed a strong interest in working closely with professors on research projects involving severe storms. It has been our experience that many atmospheric science students have a desire to learn severe storms forecasting, and apply that knowledge in ways that are beneficial to society. Sensing an opportunity to enhance student learning of atmospheric science, we created an elective “severe storms field work” course within Purdue EAPS. Learning, in this context, is defined as, “the process whereby knowledge is created through the transformation of experience” (Kolb 1984). The rationale for the creation of this course was well expressed by King (1993): “When students are engaged in actively processing information by reconstructing that information in such new and personally meaningful ways, they are far more likely to remember it and apply it in new situations.” Specifically, we hypothesized that by having students actively participate in forecasting, observing, and collecting data in severe storms in a field work context, their retention of the course material would be enhanced, they would gain valuable technical skills, and expand their career aspirations. In this study, we provide quantitative evidence that the Students of Purdue Observing Tornadic Thunderstorms for Research (SPOTTR) course is indeed having these effects on its participants.
The SPOTTR course was first piloted at Purdue in summer 2016 at the request of six undergraduate students. Three Purdue professors (authors Tanamachi, Dawson, and Dr. Michael Baldwin) chose to pilot SPOTTR during Purdue University’s 4-week “Maymester” session, which lasts from approximately 10 May–8 June, and coincides with the climatological peak for severe weather events in the central United States. As originally conceived, SPOTTR (worth one course credit) was designed to fulfill the following objectives for students:
L1) To learn current severe weather forecasting and observation techniques
L2) To have an authentic atmospheric science field work experience, using research-grade observing instruments, and opportunities to continue to work with collected data if they chose to do so
L3) To expose students to various career paths in meteorology, including paths students may not have been aware of prior to taking the course, mainly through interactions with meteorologists
L4) To enhance student learning of severe storms forecasting and research through reflective journaling and other autonomous active learning exercises
Throughout this manuscript, we refer to these learning objectives (L1–L4) to clarify mappings with course activities and conclusions.
After the conclusion of the 2016 course, some of the undergraduate participants requested to continue working with the instructors, or sought them out for closer mentorship. These mentoring activities included an undergraduate research experience that culminated in a presentation at a professional conference (Seedorf and Tanamachi 2016), and requests for letters of reference from all three of the instructors by students applying to graduate school and for employment. These developments prompted the authors to more objectively evaluate the benefits of the course from a teaching and learning perspective. Over the next three iterations of SPOTTR (2017–19), the students (18 in total) were given two surveys, one at the beginning and one at the end of the course, that were designed to assess changes in their knowledge levels, confidence in field work techniques, and career aspirations. We discuss our findings from these surveys after a review of the course design.
The SPOTTR course was designed to provide an experiential learning scenario (Kolb 1984, L2). The Kolb (1984) experiential learning cycle (ELC) model consists of four stages: 1) concrete learning, in which the learner interacts with the environment; 2) reflective thinking, in which the learner compares their experience with his or her existing knowledge; 3) abstract conceptualization, in which the learner updates his or her conceptual understanding with insights gained through reflection; and 4) active experimentation, in which the learner translates his or her updated conceptual understanding into updated set of actions (Fig. 1). The cycle is then repeated to consolidate the students’ understanding (a process Kolb calls the “experiential learning spiral” in the second edition of his 1984 text, implying that the learner’s knowledge expands with each cycle).
Severe storms forecasting and observation, which typically occurs in a daily cycle, lends itself naturally to the ELC model. In the context of this course, concrete learning (stage 1) consisted of students recording observations of storms in the field. Reflective thinking (stage 2) consisted mainly of a self-directed reflective journaling exercise (described below; L4) in which students compared the observed storm to the forecast generated by the group that morning and contemplated factors that may have led to imperfections in the forecast, such as problematic numerical weather forecasts or inaccurate internal conceptual models of storm formation (e.g., “the clash of air masses”; Schultz et al. 2014) and behavior (e.g., failure to account for deviant propagation of supercells based on hodograph curvature). Students were encouraged to discuss their insights with their peers (informal mentoring) and the instructors (formal mentoring). Peer mentoring (between students) was observed to occur spontaneously, without any imposition from the instructors. Students were additionally directed to distill their insights into “lessons learned” (abstract conceptualization, stage 3) in the reflective journal (“Reflective journaling” sidebar; L4) and to update their forecast technique the next day based upon these lessons (active experimentation, stage 4). This application of Kolb’s (1984) ELC model to the SPOTTR course is summarized in Fig. 1. By repeating this cycle over several consecutive days of the trip, it was hoped that an experiential learning “spiral” would be established that would enhance the students’ severe weather forecasting skills and confidence.
Consolidation and application of storm forecast concepts in a forecast environment with stakes.
Students were asked to keep daily journals of each day’s activities, starting the day before the field trip began. Entries were expected for the morning forecast, the meteorological rationale for selection of the target area, a timeline of events during deployments, details of the deployments, and meteorological observations. Owing to the instructors’ past experience in field programs, in which multiple days started to blur together in their memories over time, it was emphasized that the journals should be updated in near–real time and be as detailed as possible (e.g., Fig. SB1). Students were also required to dedicate half an hour at the end of each day to filling in any gaps in their logs, reflecting on the accuracy of the group’s morning forecast, and contemplating ways their forecasting technique might be improved. Students were encouraged to discuss among themselves anything that was unclear or confusing (peer mentoring). They also prepared questions to bring to the instructor at the next morning’s briefing. This reflective journaling task was rooted in the second and third stages of the Kolb (1984) experiential learning model (reflective observation and abstract conceptualization), with the intent of compelling self-directed learning within each student. In postcourse surveys, the SPOTTR students responded to the prompt, “My SPOTTR journal helped me clarify my thoughts about my experience,” with a mean score of 3.3 on a 4-point Likert scale, falling partway between “agree” and “strongly agree.”
The approach of taking atmospheric science students on an extended field excursion to “Tornado Alley” to forecast and observe severe storms is not new. The reader is referred to Godfrey et al. (2011) and Barrett and Woods (2012) for summary listings of similar courses taught at other institutions and their educational justifications. Typically, such courses contain 6–20 student participants and 2–3 instructors. The participants travel to Tornado Alley in one or more large passenger vehicles, each containing at least one experienced instructor, who guides the students in the field as they select a target storm to intercept and observe safely (Fig. 2). The principal differences between our course and previous ones is the combination of integrated research-grade meteorological instrumentation in the framework of a small severe storms field campaign (Schroeder and Weiss 2008), and active dissemination of meteorological observations obtained with that equipment to operational and research meteorologists in near–real time (Sirvatka and Stenz 2019). The course described in this study, therefore, can be considered an updated version of the courses described by Godfrey et al. (2011) and Barrett and Woods (2012).
The student participants were selected by an admission questionnaire (taken verbatim from Barrett and Woods 2012) that assessed their intrinsic motivation (Keller 2010) to learn severe storms forecasting and field work techniques, and any previous experience they had performing field work. As discussed by Barrett and Woods (2012), these questions conveyed to prospective students that the course would require active participation and contribution from the students as stakeholders, rather than passive reception of information. For most students, their primary intrinsic motivator was their individual desire to personally observe a tornado, a relatively rare and powerful atmospheric phenomenon, under the tutelage of experienced instructors. The students were also motivated to a lesser degree by other, longer-term benefits (as gleaned from their essays): opportunities to have hands-on experience deploying and operating meteorological instruments, opportunities to visit sites and meet atmospheric scientists engaged in severe storms forecasting and research (professional networking), real-time interaction with National Weather Service personnel, and the procurement of new, original datasets that the students could use in subsequent projects and classes.
During weeks 1 and 2 of the 4-week course, students engaged in an accelerated series of lessons on the ingredients-based method of severe weather forecasting (Johns and Doswell 1992) and polarimetric radar interpretation (L1). Additional preparatory activities included storm spotter training (Moller 1978), administered in person by a National Weather Service employee (L1), and compressed-gas safety training, for proper handling and dispensation of compressed helium used in the radiosondes. Students were also trained on each of the meteorological instruments to be used during the field trip, and oriented to the research objectives involving each instrument (Fig. 3a; L2). Specific observation objectives varied from year to year, depending upon the instructors’ research priorities, but consistently involved obtaining meteorological observations in and around supercells.1 The students, therefore, acted in the role of research assistants for the duration of the field trip (L2). Health, safety, and emergency procedures were emphasized consistently throughout the training period.
The SPOTTR instruments were selected for their capacity to collect research-quality (i.e., high-accuracy and high-precision) meteorological data in active severe weather field programs. Data collected during the field trip supplemented the instructors’ ongoing research programs as well as student projects (L2). Instruments used in SPOTTR from 2017 to 2019 included the following:
A mobile, volumetrically scanning, polarimetric Doppler radar for collecting volumetric observations of potentially tornadic storms.
A radiosonde system for upper-atmospheric thermodynamic and kinematic measurements. Specifically, we used a Sparv Embedded brand Windsond radio sounding system for recording soundings in the lowest 9 km AGL of the atmosphere (Figs. 3a,c). This system was selected because of its relatively low operating cost (including consumables like sondes and helium) and portability.
Portable In Situ Precipitation Stations (PIPS; Dawson et al. 2017) for measuring drop size distributions, air temperature, relative humidity, pressure, and wind speed and direction within convective storms (Fig. 3d).
Each student was also issued a Kestrel 4000 Pocket Weather Tracker, a handheld meteorological measuring device, to record impromptu surface observations in the field for their journals and logs.
Each student was expected to attain “mastery” of the Kestrels (i.e., the ability to operate the instrument from startup to shutdown with minimal to no assistance from the instructors) and at least one of the larger instruments. It was our experience that students each naturally gravitated to one particular instrument, were eager to attain mastery, and by the end of the trip, had designated among themselves primary and backup operators for each instrument. In addition to their personal reflective journals (“Reflective journaling” sidebar), students were also charged with logging each deployment (L2).
The participants then traveled west from Purdue University to the central United States—the “Tornado Alley” region climatologically favored for severe weather in May and June—for a period of seven days (week 3 of the course). Each day of the trip, the class followed a scheduled routine (Table 1) designed to emulate those used during severe weather research programs in which the instructors had previously participated, including VORTEX2 (Wurman et al. 2012) and VORTEX-Southeast (Rasmussen and Koch 2016). As in VORTEX2 (Wurman et al. 2012), the field team was untethered and nomadic. Each day, the group assessed the conditions and potential for severe weather and traveled to areas deemed favorable for supercells. The assessment was performed with rotating pairs of students preparing a weather briefing and announcing a provisional target, then opening a discussion with the other participants to evaluate the provisional and alternate targets (L1).
It could be argued that, for the abstract conceptualization stage 3 of experiential learning, the students’ target should always be chosen, so that the students could directly reap the rewards and consequences of their forecast (i.e., if the students chose the “wrong” target and therefore did not see a tornado, that consequence would be accepted, and compel an update to their forecast technique in the future). The instructors anticipated that ceding the authority for target selection to less experienced forecasters might set up a conflict with the objectives of their funded severe weather research projects. However, this issue turned out to be less problematic than the instructors anticipated. Only once or twice per trip did students’ choice of provisional target(s) differ significantly from those of the instructors. When they did, the instructors used the opportunity to bring to the students’ attention factors that may have been overlooked by the students in the preparation of their briefing, then reopened the discussion. For example, on 28 May 2019, the student forecasters selected a provisional target of Kansas City, Missouri, owing to a favorable juxtaposition of high CAPE and deep-layer shear that was forecast to be present there later in the afternoon. The instructors related their past difficulties collecting targeted meteorological observations in and near a large metropolitan area like Kansas City. Additionally, they pointed out the presence of a triple point in the more open area of north-central Kansas, west of Kansas City in an environment of overall less instability and shear, but with the potential for more discrete (and thus, more observable) storms and for locally enhanced shear along the warm front. Based on these insights, the students eventually settled on the triple-point target. The group collected observations in the inflow environment of a tornadic supercell there later in the day. This anecdote illustrates how the instructors were able to turn a potentially problematic situation into a lesson for the students, while reducing the risk to resources that were allocated for research.
When the group entered “chase mode,” instructors solicited students for input regarding where each instrument should be optimally deployed, then made the final deployment decisions. Students were tasked with preparing instruments for data collection using technical skills learned during orientation (L2). These included
1) inflating a balloon, activating the radiosonde, attaching it to the balloon, choosing a launch site, and actually launching the balloon;
2) finding appropriate deployment sites for the PIPS, placing them on a level surface in an orientation that minimized wind effects, switching them on, and verifying they were receiving power; and
3) finding suitable deployment sites for UMass LPR and removing its protective antenna cover. While the operation of UMass LPR was restricted to UMass personnel, students made suggestions for scanning strategies.
The groups then collected targeted observations with the instruments (e.g., Figs. 3c,d), logging each instrument deployment as it occurred. Log templates, based on research logs used by the instructors during previous field programs, were supplied for each instrument and were filled out by students during each deployment (L2). These logs were aggregated in an online repository and became part of the project metadata. Once these deployment tasks were completed, instructors engaged the students on what they were observing in the field, asking them to identify salient storm features, such as updrafts and wall clouds, and describe their significance (see “Hotwash” sidebar; L1).
Consolidate understanding of concepts related to storm behavior based on immediate or very recent observations.
Immediate quizzing and/or debriefing of students occurs in the field while severe weather is ongoing or has just recently occurred (i.e., within a few minutes or hours). This activity was considered intermediate between the field work and the reflective journaling performed by the students in the evenings. There is no fixed format for this activity. Often, it was improvised opportunistically by the instructors:
Example 1: Students are asked to take a photo (using a smartphone or tablet) of a storm that the group is currently observing, and then annotate, for example, the updraft and downdraft regions using the device’s built-in photo editing tools. Instructors then give the students feedback on their annotations and the students adjust them as needed. Subsequently, the students can observe how the configuration of these features changes in time as the storm progresses through its life cycle.
Example 2: An instructor asks students to begin making observations of temperature, humidity, and wind with handheld weather instruments as a supercell gust front passes over the group. The instructor may ask students to estimate the temperature or velocity gradient across the boundary. This activity consolidates the conceptual model of the surface flow and thermodynamic gradients around a supercell, which had been previously covered in class using diagrams.
On days when convective storm potential was considered negligible by the forecast team, or any target areas were prohibitively distant from the group’s morning location (>1,000 km), the group would instead attempt to visit locations of interest to the students’ professional development; for example, the National Weather Center in Norman, Oklahoma (Fig. 4a). This activity is discussed in greater detail in the next section.
The course participants returned to Purdue University at the conclusion of the field trip. During week 4, the final week of the course, students worked in pairs to perform retrospective case studies of one of the days of the trip for which they had provided the morning briefing. Each pair was asked to articulate which aspects of their forecasts were accurate and which were not, what happened that day that had met or defied their expectations, and what they learned about the complexities of severe storms forecasting and intercept activities (L1). Students were encouraged to draw heavily on their reflective journals, logs, and data to explain their reasoning. When possible, students included data that the SPOTTR group had collected in the field—for example, comparing a forecast hodograph from a morning model run to one that the group collected using a radiosonde, or comparing data collected by UMass LPR to observations from a nearby NWS radar (L2). The final activity undertaken during week 4 was an active exploration of career options in meteorology (L3). Details of this activity are provided in in “Career gallery walk” sidebar.
Exploration of students’ career aspirations and exposure to potential career paths in meteorology.
A “gallery walk” (Kolodner 2002; Francek 2017) is employed in two rounds. In the first round, the students were given a series of prompts at the top of a whiteboard, to which they were to respond in writing underneath. The prompts included questions such as “What are your current career goals?”, “What skills do you require in order to achieve your goals?”, and “What is your personal definition of ‘work–life balance?’” As the students rotated through the prompts, they contributed not only their own responses, but constructive comments on other’s responses. The group then discussed the responses, with the instructors moderating the discussion and providing an early to midcareer perspective.
The second round exposed students to 15 brief professional biographies (Fig. SB2), contributed voluntarily by persons who received terminal degrees in atmospheric science or meteorology approximately 10 years ago. The instructors attempted to capture an inclusive sampling of individuals, spanning traditional and nontraditional career tracks (including some who left the field), and diverse perspectives (race, gender, and LGBTQ categories). In 2017 and 2018, students were asked to write questions in response to the biographies, which were then relayed to the biography subjects via e-mail. In 2019, in response to student feedback, students instead read the biographies, then voted on which individuals they wished to see included in a virtual career panel. Three individuals took part in the panel via video chat, and the students were able to interact with them more directly.
Beneficial class activities
Interactions with professional meteorologists.
Some of the class activities had immediate or nearly immediate benefits for both the SPOTTR students and other entities. Foremost among these was near-real-time interaction between SPOTTR participants and operational forecasters via social media.
A Twitter account (@eaps_spottr), registered with Purdue University, served as the official public communication point for the class. The account profile page featured a photo of the course participants and a brief description outlining the group’s mission. Instructors had the ability to send original tweets and to retweet posts deemed to be of interest to the group and their followers. Posts consisted mainly of photographs taken in the field, and screenshots of collected data, such as radar images or skew T–logp diagrams of soundings (e.g., Fig. 5). By the conclusion of the 2019 field trip, the @eaps_spottr Twitter feed had more than 220 followers. In 2019, the instructors also enlisted a professional writer and photographer to create a weblog (blog) showing the group’s activities leading up to and during the field trip (Carson and Uhlman 2019). These activities were supported under the “broader impacts” (improving public scientific literacy) section of the first author’s National Science Foundation grant.
The @eaps_spottr Twitter feed was used to disseminate data collected by the group within minutes of its collection. When appropriate, nearby NWS offices and other interested entities such as the NWS Storm Prediction Center and NOAA Weather Prediction Center were tagged in class postings to draw their attention to the supplemental information collected by the students. NWS personnel have consistently indicated that they find such supplemental data useful (e.g., R. Smith 2020, personal communication). For example, when the group launched a radiosonde, a skew T–logP diagram of the resulting sounding would be posted with tags for the nearest two or three NWS offices. Several NWS offices acknowledged the SPOTTR group’s tweets by replying to them or “liking” them (e.g., Fig. 5; L2). Some actively solicited the group for additional information or data, which the group provided when possible.
The Twitter feed also aided in impromptu collaborations between the SPOTTR group and other atmospheric research groups concurrently in the field. For example, during the 2017 trip, the SPOTTR group traveled to northern Alabama to intercept potentially tornadic storms spawned by the remnants of Tropical Storm Cindy. En route, instructor Dawson used Twitter to contact the University of Alabama in Huntsville’s atmospheric science research group, with whom he had collaborated previously on VORTEX-Southeast data collection. This interaction resulted in a loosely coordinated deployment of the two groups in this event. As another example, the 2019 SPOTTR group sought coordination with the Targeted Observations by Radar and Unmanned Aerial Systems (UAS) of Supercells (TORUS; www.nssl.noaa.gov/projects/torus/), and were able to provide supplemental radiosonde observations to that group (Fig. 3c). Students thereby experienced helping operational forecasters and other atmospheric research scientists working on severe weather events, as well as witnessing atmospheric science field experiments in action (L2, L4).
Professional development and networking.
One of the primary objectives of the SPOTTR class was to expose students to the various career paths in meteorology (L3). Each group was able to visit the National Weather Center in Norman, Oklahoma (e.g., Fig. 4a; L3). As availability allowed, some groups also visited other venues like the University of Alabama in Huntsville’s Severe Weather Institute Radar and Lightning Laboratory (SWIRLL) building, and the University of Oklahoma Advanced Radar Research Center (OU-ARRC; L3). The group also received invitations to visit individual NWS field offices, usually through social media interactions with those offices.
The students also connected with professional meteorologists in less formal ways. Chance encounters with other meteorologists and storm chasers commonly occurred in the field (e.g., Fig. 4b). Occasionally, serendipitous joint briefings were held with severe storms field course participants from other universities (e.g., Fig. 4c; L1, L3). The instructors also arranged for meals and other informal interactions with meteorologists when their travels took them close by (L3). Anecdotally, students reported that those interactions were enjoyable and empowering.
The meteorological and educational data collected during SPOTTR has advanced research projects for both the instructors and students. Results have been disseminated at professional conferences (Seedorf and Tanamachi 2016; Tanamachi 2018; Tanamachi et al. 2016, 2018a,b, 2019a,b; L2), some with student participation, and multiple formal manuscripts are in preparation at the time of this writing.
SPOTTR pre- and postcourse surveys
Survey instrument description.
In the interest of quantifying the perceived benefits of the course to students, we gave the 2017–19 SPOTTR students two surveys during the course, one at the beginning (“pre”) and one at the end (“post”). The survey was a modified version of the instrument used by Adedokun et al. (2014), which was designed to assess changes in students’ research skills, confidence in field work techniques, and career aspirations as a result of participating in an undergraduate research experience. We modified some of the questions to refer specifically to aspects of the SPOTTR course. Some of the questions asked students to rate themselves or indicate agreement with a statement on a four- or five-point Likert scale. Other questions required free-form text responses. Some key questions from this survey are summarized in Tables 2–5. Additionally, the 2018 and 2019 class participants were given the “Content and context quiz” of Barrett and Woods (2012, their Table 5) in order to assess changes in their knowledge levels.
The dataset thus generated comprises a set of 19 responses by 18 unique students. The reason for the discrepancy is that one student repeated the course in both 2018 and 2019. Because the survey results were anonymous, it is not possible to isolate and link that student’s responses to one another. We therefore treat it as a separate, independent response (and plan to tag responses from repeat students in the future). Additionally, owing to a survey coding error, some of the paired questions were not included in the precourse survey given to the eight SPOTTR students in 2018. Any postsurvey responses without matching presurvey responses were excluded from this analysis. We have annotated the response rate n in all tables in order to show how many paired responses were analyzed.
Of these 18 SPOTTR students, 7 were graduate students, 10 were undergraduates, and 1 was a non-degree-seeking student who had previously earned a terminal degree in atmospheric science. Seven students identified as female, and 11 as male. Fourteen of the students specialized or majored in atmospheric science (including the non-degree-seeking student), and four specialized in non–atmospheric science disciplines, demonstrating SPOTTR’s appeal outside meteorology. (The SPOTTR course was open to students from any major with junior standing or higher.)
Owing to the limited sample size, the findings are not generalizable to all populations of atmospheric science students. However, the findings may be transferable to other teaching contexts with similar student populations. The findings presented here should be considered one possible outcome from this type of learning environment; results can be expected to vary depending on numerous factors, such as students’ backgrounds and instructors’ individual teaching philosophies. To make the results more applicable across a range of educational settings, we have attempted to pinpoint those factors that were most beneficial for the students.
Gains and losses (pre- to postcourse paired questions)
For those questions with numeric (four- or five-point Likert scale) responses, it is possible to quantify the changes in students’ responses between the beginning and end of the course. While one can simply subtract the pre- from the postsurvey responses and then take the mean, this mean change metric fails to adequately capture the effects on individual students. For this reason, we also calculated normalized change c metric (Marx and Cummings 2007) to quantify the impact of the SPOTTR course on students. The c metric, which is calculated for each individual student, is defined as
Here, c is formulated to account for these possible cases:
a student experiences gains by taking part in the course, in which case c is equal to the normalized gain (Hake 1998);
a student has already attained the maximum score for the question at the beginning of the course, in which case the student falls outside the scope of the instrument, and their response is dropped;
a student exhibits no change, in which chase the change c is 0;
a student experiences losses by taking part in the course, in which case the loss is normalized by their presurvey response (i.e., the maximum possible loss).
In essence, c, which ranges from −1 to +1, is designed to answer the question, “For those students who experienced a change as a result of taking the course, what is the relative magnitude and sign of that change?” The normalized change is interpreted as “small” if the magnitude of c falls in the range (0.0, 0.2], “medium” if c falls in the range (0.2, 0.8], and “large” if c falls in the range (0.8, 1.0] (Marx and Cummings 2007).
We found that c was more revealing than the mean change in numerous instances. For example, the mean change in response to the question, “To what extent do you agree or disagree with the following statement? College faculty can encourage and promote my interest in pursuing a research oriented field,” was 0.0 points on a five-point scale, but the normalized change c was +0.5, indicating a medium positive impact (Table 2). In other words, those students who experienced a change as a result of taking the SPOTTR course experienced a medium level of benefit, even though the mean change potentially indicated no gain. We emphasize again that although mean c is reported tables below, c was calculated for each individual student, for each question.
For those paired questions pertaining to student aspirations (Table 2), c indicates majority positive impacts across the board. In particular, SPOTTR students reported borderline medium–large positive impacts on their awareness of what graduate school may be like, the various career opportunities available to them, and the career paths of the faculty with whom they worked (L3). Medium positive impacts were recorded for questions pertaining to student’s aspirations to attend graduate school, confidence in pursuing future careers involving research, and awareness of career options (L3). Overall, the SPOTTR course was found to have substantial positive impacts on undergraduate students’ ability to envision themselves as future graduate students, and on all students’ ability to envision themselves as potential researchers.
Responses to paired questions specifically related to skills needed to perform research in severe weather scenarios (Table 3) showed even larger positive change overall, likely owing to the specific nature of the training provided and the real-world application of those skills. In particular, those students who experienced a change as a result of taking part in the SPOTTR course reported large positive normalized change (c = +0.9) in understanding of how to collect meteorological data safely in severe weather, and borderline medium–large normalized change (c = +0.8) in their understanding of how to operate meteorological equipment, coordinate with other teams in the field, and stay safe near severe weather (L2). Normalized change for the remaining questions fell in the high end of the “medium” range (i.e., +0.5 to +0.7). It can also be seen from swarm plots of the responses (Figs. 6 and 7) that students self-reported “climbing the ladder” one or two levels for almost all of the skills assessed.
The above results pertain to self-reported levels of knowledge gain. After the 2017 SPOTTR survey, the authors felt that there was a need to measure knowledge gain more objectively. Starting in 2018, the “content and context” instrument of Barrett and Woods (2012, their Table 5) was added to the pre- and postcourse surveys. This brief knowledge inventory poses questions regarding basic meteorological terms and concepts related to severe storms forecasting, as well as an analysis question (Table 4; L1). Responses were scored by the instructors out of 100 possible points.
As with the other pre–post paired questions, mean and normalized change were computed from the knowledge inventory scores (Table 4). For those students who experienced a change as a result of taking the course, the greatest knowledge gains were exhibited on questions pertaining to supercells (4.3 and 4.4), and on the analysis question. In particular, students exhibited large positive normalized change on question 4.4, “What are the atmospheric ingredients necessary for supercell formation?” (c = +0.9). Borderline medium–large normalized change was exhibited on both question 4.5, “What is a supercell?” (c = +0.8) and on question 4.7, which required application and analysis of a forecasting scenario (c = +0.8). Medium changes were exhibited for questions pertaining to tornadoes (questions 4.5 and 4.6) and the low-level jet, a key mesoscale feature of many severe storm environments (question 4.2). It is evident from these results that the students’ gained a firmer conceptual understanding of supercells, tornadoes, and their larger-scale environments, as well as how forecasters predict their formation, as a result of taking part in the SPOTTR course (L1).
To prioritize which elements of the course to retain in the future, the postcourse survey asked students to rate the value of certain course elements on a five-point Likert scale (Table 5). Students could give ratings ranging from 1 (“Not at all”) to 5 (“A great deal”). Almost all of the elements had mean ratings greater than 3.0, meaning almost all students valued the elements from “a moderate amount” to “a great deal.” In the mean, students rated daily forecasting as the most valuable element of the course (4.6), followed by the retrospective case study (4.4). These two results are desirable because they addresses the symmetric processes of forecasting and forecast assessment (L1). Field deployments of instruments (4.3; L2), and visits to sites like the National Weather Center (4.2; L3) also rated highly. A few outliers (not shown) rated the value of elements related to professional development as either 1 or 2; inspection of associated written responses revealed that at least one of these respondents was from outside the atmospheric science discipline. Therefore, it is reasonable that this person did not derive as much value from, for example, visiting the National Weather Center as the atmospheric science students did.
Student satisfaction with the course was gauged via a series of statements to which students indicated agreement or disagreement on a four-point Likert scale (Table 5). The responses were overwhelmingly positive, with mean scores all greater than or equal to 3.3. For example, the statement, “I found the SPOTTR course to be a positive learning experience,” garnered a mean response of 3.8 (close to “strongly agree”), and the statement, “The use of research-grade equipment enhanced my research experience” (L2), garnered a mean response of 3.6 (between “agree” and “strongly agree”) on the four-point scale. One statement was phrased negatively: “I did little or nothing that seemed to me to be real research,” with a mean response of 1.6, falling between “strongly disagree” and “disagree.” Thus, for this statement, a “disagree” response is analogous to an “agree” (positive) response for the other questions. In the aggregate, the students’ responses indicate a high level of satisfaction with their experience in the course, a sense of their community with their peers, and an enhanced understanding of the research process (L2).
Comments from students
Not all of the impacts of SPOTTR could be expressed on a Likert scale. Free-form text responses were solicited from students in the postcourse surveys to the following questions.
What important lessons did you learn from the SPOTTR course? A word cloud representation of the text responses is shown in Fig. 8a. A few responses to these questions named specific skills (e.g., accessing and interpreting NWP models, soundings, and radar data). However, many responses related to the advancement of higher-order thinking skills such as analysis, evaluation, and creation (Bloom et al. 1956). Recurring themes included the following:
Flexibility in forecasting and field operations (L1, L2): “Set schedules can be great for planning forward, but when plans fall apart, spontaneity and open-mindedness make a person more adaptable to a variety of situations,” and, “I was amazed at how many time[s] we had to reassess our target because models changed and so did the forecast.”
Students’ self-conceptualization as future professionals (L3): “[I am] More clear about my future career,” and “Hopefully in the future I can eventually be able to have my own field projects that I lead.”
Increased awareness of meteorological career paths and opportunities (L3): “I learned a lot about different career paths that can be taken as a meteorologist, specifically when it comes to research,” and “There are countless more jobs [within] the meteorology field than I thought.”
Students learned about their own metacognition: “I was also able to grasp concepts that I learned in class better than before the class. For example, seeing and feeling the different downdrafts, inflow, and outflow of the storms, allowed me to understand those concepts better.”
What important lessons did you learn from the journals you kept? A word cloud representation of the text responses is shown in Fig. 8b. By far, the predominant recurring theme was the need to update journals in real time as often as possible, in order to keep events and details organized (L4). Examples of such comments include the following:
“It is important to take notes on the spot. I had written my journals at the end of the day, and unfortunately I likely forgot several key details of some of our daily events.”
“We tend to forget the fine details about day’s happenings since plans change so quickly. Maintaining a journal helps keep… a sanity check about impromptu actions taken during the chase.”
“Be detailed. Don’t forget to fill these out ASAP. Everyone is going to remember details differently.”
Multiple students also related that the journals helped them understand the dynamic nature of the forecasting process, and empowered them to learn more effective forecasting skills (L1):
“Because there are often multiple storms that develop, [journaling] helps with reflection on the day, and analyzing what went wrong with the storm that was actually targeted.”
“Forecasts are never going to be perfect. Look at the general idea of NWP forecasts rather than the small-scale details.”
“Even with proper preparation, it’s very possible that there are days chasers may be shut out, or choose the incorrect storm to follow, so the reflection done in the journal is useful for determining what went wrong that day, and how to improve on forecasting in the future.”
This last comment in particular ties back strongly to the ELC-based design of the course, and foreshadows the self-directed learning skills exhibited by successful professional meteorologists (LaDue and Cohen 2018).
How do you think the overall SPOTTR course can be improved? Most students conveyed enthusiasm for the overall course design. About half the students expressed a desire for the field trip to be longer than seven days, and about a third for the in-class portion of the course to be given over a longer period. Budgetary and scheduling constraints have thus far prevented us from offering a longer version of this course. However, the authors envision expanding this course into a multicourse sequence in the future in order to reinforce and expand the learning objectives.
Additionally, students returned mixed commentary regarding how they would like to see the meteorological instrumentation used in future iterations of the course. One wanted to use the instruments more (“[I suggest] some sort of semi-‘forced’ rotation of instruments”), while another suggested that we scale back their use (“[I suggest] we focus more on forecasting and maybe one or two research instruments (not three).”) However, based upon the response to question 5.19 (Table 5), which clearly indicates that the students perceive that experience fielding instruments is valuable (L2), we intend to continue using meteorological instruments in this course.
Please provide any additional comments you would like to share. As might be expected, responses to this solicitation varied widely in focus, including safety (“I never felt particularly in danger during the pursuit of any storms”), professional development (“This class really provided me some great things to add to my resume for my future”), and patience (“The long trip was very tiring”). Multiple students felt that they had learned more about severe weather from a week in the field than from multiple weeks spent in the classroom (L1). One wrote, “This class is great, not necessarily to learn brand new topics, but can help visual learn[er]s like me understand what I am being taught in my classes a lot better than just reading the same thing over and over in books.” Multiple responses also related increased confidence. One wrote, “I feel much more confident in my ability to understand advanced papers and presentations about severe weather research.”
A new course (SPOTTR) was created at Purdue University that leveraged students’ desires to observe severe weather and gain research experience under the guidance of knowledgeable instructors. The course incorporated research-grade meteorological instrumentation and hypothesis testing in order to give the students an authentic experiential learning scenario, based on the instructors’ experiences in past severe weather research programs. The course design was based on the Kolb (1984) ELC active learning model, with all four stages of an ELC occurring on each day of the trip.
Through a study incorporating survey responses, we have shown that this course design is effective for improving both students’ self-reported skills and objectively evaluated knowledge inventory. In particular, students reported substantial (medium to large) gains in skills related to severe weather forecasting, deploying meteorological instruments safely in severe weather, and logging deployments (Table 3). Focusing on students’ longer-term goals, students reported substantial (again, medium to large) gains in their respective levels of confidence in understanding research procedures, expanded awareness of career paths available to them, and an increased desire to pursue graduate education. It is evident from these results that the SPOTTR course has helped students to more clearly envision themselves as potential future graduate students, researchers, and/or forecasters. All four of the course objectives (L1–L4) were achieved for most students.
Perhaps the most remarkable aspect of this study is that these gains occurred over a period of only four weeks. While the SPOTTR course design was intensive and required a substantial investment of time and resources from the department, instructors, and students, our results suggest that these investments are worthwhile, given the potential long-lasting benefits.
The survey we formulated, however, only offered limited insight into why the course had such positive effects on students. Additionally, since the course format was mostly consistent over the three years of this study (2017–19), we lack a “control” group against which to assess the impacts of individual course elements. In the absence of such objective metrics, we offer some informed speculation as to why the SPOTTR course was effective.
First and foremost, we attribute the benefit to the cohesion that was observed by the instructors within each SPOTTR cohort. Each cohort spent more than 150 h traveling together, eating together, talking together, and lodging together, creating a shared experience base and a common reference frame. This educational setting increased the likelihood of both formal (peer mentoring) and informal interactions between group members. It also afforded opportunities for the instructors to talk to individual students at length about extracurricular issues, such as applying to graduate school, and “long view” topics like work–life balance. The instructors assigned weather briefing pairings between more and less experienced forecasters, thereby providing each member of the “less experienced” group with a ready-made peer mentor. Additionally, the group was collectively dedicated to a shared scientific mission. The instructors experienced similar long-term cohesion among participants in previous field projects (e.g., VORTEX2), which also involved a shared mission and common experiences.
Second, we emphasize that it was Purdue students, not the instructors, who initially requested that the SPOTTR course be created (see first section). Purdue University is located relatively far away from the Great Plains, in northwest Indiana. Student participation in severe weather field experiments is not as commonplace as it is at other universities located on the Great Plains. Students, in this instance, exercised their autonomy to create an educational opportunity for themselves, and were therefore invested in their personal and professional growth as an outcome. In other words, the student population self-selected for those who were more likely to benefit from activities that helped them to envision themselves as future graduate students and/or research scientists.
Of course, field work is only one stage of an atmospheric scientific research project. In the future, we envision offering an expanded version of SPOTTR consisting of a spring–summer–fall sequence of three courses aimed at providing a more complete atmospheric research experience. The first (spring) course will focus on developing working hypotheses and designing experiments related to severe weather, the second (field work) will apply those experiment designs in the field as described herein, and the third (fall course) will consist of data analysis and preparation of conference presentations based on the data collected in the second course. The summer SPOTTR course will thereby serve as the linchpin in a directed course sequence, providing focused professional preparation to atmospheric science students hoping to enter graduate school and pursue research-oriented careers.
The majority of the funding for the SPOTTR course from 2016 to 2019 was provided by the Purdue EAPS department. We are grateful to our coinstructor, Prof. Michael Baldwin, for helping us pilot SPOTTR in 2016. The Purdue College of Science sponsored the inclusion of the UMass X-Pol radar in 2016 and 2017 through a research startup grant to the first author. Use of the UMass LPR in 2018 and 2019 and the live blog for the 2019 field trip were both supported by NSF Grant AGS-1741003 to the first author. The blog was authored by Mary Kay Carson and Tom Uhlman, who provided some of the images used in this manuscript. Steve Frasier, William Heberling, and Carl Wolsieffer (all UMass) supported deployments of the UMass radars. Michael Biggerstaff and Sean Waugh developed the PIPS probes in collaboration with the second author. The acronym SPOTTR was suggested by student Matthew Seedorf. Sam Lashley, of the NWS forecast office for Northern Indiana, voluntarily administered the storm spotter training. Vittorio (Victor) Gensini provided course materials from the College of DuPage storm chasing class. The authors wish to thank the volunteer tour guides of the National Weather Center (Pat Hyland), the OU RIL (David Bodine), and UAH SWIRLL [Anthony (Tony) Lyza] for allowing the SPOTTR group to visit their facilities, sometimes on short notice. We are immensely grateful to the 18 SPOTTR students who consented to participate in this research (Purdue IRB 1704019125). Last, we express our appreciation to Dr. Bradford Barrett and two anonymous reviewers, whose insightful comments resulted in substantive improvements to this manuscript.
For example, the first author was funded by NSF to collect polarimetric radar observations of supercells in 2018 and 2019, incorporating the SPOTTR course as an educational component.