A new laboratory course provides sophomores an opportunity to learn the tools and methods of research, think critically about new ideas, practice disseminating results, and clarify career choices.
Research experiences for undergraduate students provide opportunities to increase their understanding of the process of creating new scientific knowledge, to increase their ability to overcome obstacles, to develop a better knowledge of available careers, and to expand their interest in postgraduate studies (Lopatto 2004, 2007; Seymour et al. 2004). These findings have been primarily published in the science education literature and include participants from many different research fields (chemistry, biology, and at times even the social sciences). Research experiences for undergraduates have also existed in the atmospheric science community for several decades (e.g., Rutledge et al. 1993; Newman 2008), but the structure of the programs and their outcomes are published far less. A more recent study by Gonzalez-Espada and LaDue (2006) reports benefits to students who have participated in the National Weather Center Research Experience for Undergraduates program and identifies similar outcomes as those among other disciplines, including an increased interest in attending graduate school and a clarification of their career choices.
Unfortunately, these benefits are often limited to a few upper-level students who work within faculty members' research groups during the academic year, have the opportunity to take a special topics or independent study course, or are able to secure a place in the small number of highly competitive summer atmospheric science research programs offered at various institutions across the country [e.g., Significant Opportunities in Atmospheric Research and Science (SOARS; Windham et al. 2004) and the McNair program (Ivanova 2008)]. These experiences tend to be predominantly designed for seniors or sometimes juniors and seniors. Fortunately, the inclusion of research experiences in the undergraduate classroom is becoming more popular (e.g., Schroeder and Weiss 2008; Morss and Zhang 2008), sometimes because of extra support from larger agencies (e.g., Unidata's support for inclusion of research experiences in the undergraduate curriculum; Etherton et al. 2011), but these types of courses are still not an integral part of many undergraduate programs.
The importance of providing the benefits of research experiences to all undergraduate students and earlier in their academic program motivated the development of a sophomore-level course in atmospheric science research, based upon a successful model used in chemistry courses, designed around an authentic research project. A recent meta-analysis of evaluations of undergraduate research experiences by Sadler et al. (2010) found that, when students are involved in these opportunities over a longer period of time (as opposed to a single summer), they become more comfortable in their roles as researchers. This confidence allows the student to be more active in the research process, amplifying the positive outcomes of the experience. Lopatto et al. (2008) has also shown that when these experiences are integrated into structured courses, students still report many of the same benefits as other research experiences.
We report here upon a new research course designed as a first research experience for sophomore majors in an atmospheric science program.1 We first describe the course structure, addressing the challenges of completing a research project within a classroom setting and of tailoring projects for students early in their undergraduate programs who have limited knowledge of atmospheric science. We then present a formal evaluation of the student outcomes, as analyzed through standard qualitative methods in science education research, using interviews with students, student work, and a postcourse survey (the survey is available online at http://dx.doi.org/10.1175/BAMS-D-11-00061.2).
COURSE DESIGN.
The course structure was based upon that of the Center for Authentic Science Practice in Education (CASPiE; Russell et al. 2010), originally created for freshmen chemistry laboratories. The CASPiE design requires a research scientist to develop a research “module” that includes prelaboratory readings, laboratory activities, and postlaboratory assignments. Each module is separated into three sections consisting of 1) introductory material, including background information and skills the students need to perform the research project; 2) the research methods, data acquisition and analysis, and new results that are useable by the research module author; and 3) an open-inquiry project, where students develop and execute their own research idea that extends the original project. Typically, two research modules are completed during a semester course. Throughout the modules, students complete inquiry-based lessons, where the teacher acts as a facilitator of knowledge and students direct the learning during the course (Anderson 2002). The second and third sections of each module consist of research activities, resulting in two-thirds of the course being spent on authentic research practices. Students learn to collaborate with peers, develop and test hypotheses, construct procedures, draw conclusions, and present their work while overcoming the obstacles that are characteristic of authentic research projects. Student grades are based upon introductory exercises, completion of research tasks, written formal lab reports, and final presentations. The research tasks produce results that are unique for each team of students. Thus, students are assessed on their ability to describe the experiment procedures and their theoretical support and to create a logical and well-designed presentation of the data and results.
Two changes were made to the CASPiE model to tailor it for atmospheric science. Chemistry modules dedicate a significant portion of their time to learning specific techniques, such as making solutions. For atmospheric science research, students could master laboratory techniques (e.g., running software or models) rather quickly; however, they needed extra time to assimilate new concepts and ideas due to the early stage of the students' undergraduate career (i.e., sophomore year). The CASPiE model was thus adapted by dedicating more time to concepts in the first few weeks and reinforcing these concepts throughout the module. The second change to the CASPiE model addressed differences in the number of students enrolled. CASPiE is designed to be taught completely by teaching assistants (TAs), for class enrollments of hundreds of students. Because of the smaller class size typical of atmospheric science (for this course, an enrollment of 18), the module author could attend some class meetings to guide students through the research project and provide expert advice. The availability of the module author allowed for small adjustments to the module to be made (e.g., modification of the analysis technique as a result of problems the students encountered) in real time as the research project progressed. Although this type of adjustment to the research project was not originally designed into the structure of the course, the process of implementing the modules for the first time made their necessity apparent. The smaller class size also allowed the teaching assistant more time to work with students who needed more assistance or wanted further discussion.
MODULE DESIGN.
Students completed two modules during the course. The basic structure of the modules was similar, but each focused on a unique research project with related concepts and skills. The module topics and the required skills covered a wide range, giving the students a broader overview and more variety in their research experience. Students worked in teams of two or three. Anecdotally, we observed that students who were in groups of two appeared more able to keep each other informed of their progress and to split responsibilities.
New concepts.
In the first weeks of each module, students learned new concepts using traditional atmospheric science activities, such as map analysis, to understand synoptic weather patterns or skew T analysis to understand the vertical properties of the atmosphere, and applied the new concepts to the module project. Here, the module authors used the principle of “scaffolding,” where students are given instructional support when a new concept or skill is introduced, and then they gradually proceed more independently as they gain knowledge and expertise (Wood et al. 1976). An example is the activity used to teach students about the warm-rain process (Fig. 1). Students observed the changes on the surface of a cold can of soda on a warm, humid day to visualize the warm rain process (condensation on the side of the can, where some drops grow larger than the others and slide down the can side, collecting other droplets and growing, and also then moving faster down the side of the can mimicking collision coalescence). During this process, students were given suggestions for the analogy to the warm rain process and questions to lead them to understand the experiment. At the end of the observations, a class discussion helped students articulate and reflect upon their developing knowledge. Later, during the research project, students were expected to draw conclusions using this knowledge applied to the output from a cloud microphysics model. Thus, as students progressed through the research module, more time was used to apply and extend their conceptual understanding of phenomena and less time introducing new concepts. As in real research, students proceeded through the project in a nonlinear manner: they learned from mistakes, revisited earlier decisions, and made appropriate adjustments to complete the research project.
Visualization of the interplay and feedback among student learning activities concerning the warm-rain process throughout the course.
Citation: Bulletin of the American Meteorological Society 93, 11; 10.1175/BAMS-D-11-00061.1
New techniques, tools, and skills.
Early on, it was important to create learning activities to give students enough practice (i.e., using some standard meteorological analysis techniques with the new concepts they had learned) necessary to understand and complete the research project. As an example, a curriculum map of the development of content knowledge and skills for the first module is shown in Fig. 2. In addition to the warm rain process, students also learned to analyze a thermodynamic sounding in order to understand and operate the cloud microphysics model that was initialized with a sounding. Students worked with an “adapted skew T chart” (Fig. 3) where only the isobars and isotherms were shown. In later meetings of the class, the module author and TA did the necessary “parcel analysis” and taught the students to interpret cloud-base and cloud-top heights (required inputs to the microphysical model). In this manner, students were not overwhelmed by having to learn the complexities of a full skew T diagram in such a short amount of time.2 In addition, background material on important differences in maritime and continental clouds (Fig. 4) provided the content that students would use later when interpreting why their modeling results in various locations across the United States could be different but still correct. Students were then ready to make inferences regarding cloud condensation nuclei (CCN) differences affecting outcomes of the warm rain process in their numerical modeling experiments.
Sample curriculum map of concepts and skills learned in the first three weeks of module 1 and their relation to the course objective of understanding and operating the microphysics model. Green, purple, and orange boxes represent weeks 1, 2, and 3, respectively.
Citation: Bulletin of the American Meteorological Society 93, 11; 10.1175/BAMS-D-11-00061.1
Excerpt from the introductory reading for module 1, where students learn to interpret temperature profiles of the environment (red curve) and of a lifted parcel (blue curve) on a simplified skew T chart.
Citation: Bulletin of the American Meteorological Society 93, 11; 10.1175/BAMS-D-11-00061.1
Excerpt from the introductory reading for module 1, where students learned how CCN differences in (bottom) continental and (top) maritime environments lead to differences in cloud droplet number concentrations and droplet sizes and then could discuss implications upon the overall warm rain process, whereby the maritime clouds produce rain more quickly.
Citation: Bulletin of the American Meteorological Society 93, 11; 10.1175/BAMS-D-11-00061.1
Impediments to learning.
During the first implementation of the module, it was discovered that students were not familiar with the computer interface to understand exactly what their Unix-style commands were accomplishing. Most undergraduates in atmospheric science programs have not yet had a scientific programming class the first semester of their sophomore year. For example, students had difficulty correctly saving output files and transferring them to a “home” directory for future analysis, which resulted in loss of work in the first week. Later offerings of the course include computer skills in the introductory labs to prepare the students better for computational activities required by the research.
Student-based inquiry.
During the second portion of each module, students designed related experiments, through a discussion facilitated and guided by the module author. The module author provided useful information only after the students had time to think carefully about topics and only as necessary to keep them from deviating too far from the objective. Students had typical concerns, such as statistical significance regarding the number of cases and the need to explore a larger parameter space and thus asked to perform more experiments. Some were intrigued by simplifying assumptions made earlier in the project and designed experiments to explore their implications. Although these additional experiments were of varying degrees of usefulness to the overall project, they provided students the opportunity to witness the value of creativity in scientific research and the many “dead ends” to which authentic research can lead before reaching a final result.
Formal scientific communication.
Finally, each module concluded with a class discussion of the results and the creation of a poster and slideshow to present to faculty and graduate students at the last class meeting. Guidelines for giving effective presentations were distributed to the class and discussed, and students rehearsed the presentations. Students participated in an oral or poster presentation and were expected to respond to questions from the audience to simulate what would be expected at a professional meeting. Two students later presented the class results at the Ninth Annual Student Conference of the 90th Annual Meeting of the American Meteorological Society (AMS) (Fig. 5). A student's ability to clearly articulate the research project and respond to impromptu questions was considered a significant indicator of his or her level of comprehension.
Two students from the 2010 implementation of the course present the class posters at the AMS student conference in Jan 2011. Photos from the 2010 conference were not included to protect the confidentiality of students who participated in the evaluation of the course.
Citation: Bulletin of the American Meteorological Society 93, 11; 10.1175/BAMS-D-11-00061.1
Implementation: Module design.
Some research projects are more appropriate for adaptation to the classroom than others. The content of the research project must be accessible after only a few weeks of introductory material and ideally uses scaffolding techniques. The best modules have activities that need to be performed many times, need to be performed for many cases (e.g., requiring groups of students to run models many times over a large parameter space for sensitivity studies), or that analyze meteorological data for many cases in order to build up a statistically defendable result. These studies allow students to be working on projects with the same theoretical background that can be taught in the early meetings of the course. Modules must consider the length of the semester, the size of the class, and the initial skills of the students.3 The length of the semester necessarily limits the scope of the project; students complete a portion of a publishable research project and contribute to future work of the module author. The level of scientific contribution by the students was similar to what would be expected from a student during other forms of undergraduate research (Lopatto 2009). The majority of the project was completed within the formal class time (a weekly 3-h class period), but students often needed extra time outside of class to complete the objectives.
Implementation: Examples of research modules.
The first module consisted of cloud microphysics research. The module author was studying variations in the warm rain process due to projected changes in the vertical structure of the atmosphere resulting from regional climate change. The goal of the students' research was to run the model (Cooper et al. 1997) to investigate cloud behavior over many locations throughout the United States, compile the results, and identify any regional patterns of change in the warm rain process and/or geographical areas of interest for future research with more sophisticated models. In this module (outlined in Table 1), students needed not only an introduction to some basic meteorological and numerical modeling concepts but also skill development in using spreadsheet software to organize and graph data and in using Unix-style commands to run the model.
Warm rain process in a changing climate module.
The second module focused on severe weather research. The module author was interested in investigating the possible upscaling effects of mesocyclones on the synoptic environment. The goal of the research project was to compare the location of supercells as identified with Doppler radar with the initialization fields for the North American Model (NAM) to look for sinks or sources of vorticity or convective available potential energy (CAPE). This module focused upon analyses of a numerical weather prediction model initialization field and raw radar data. For this module, students learned to use the Integrated Data Viewer (IDV; Murray et al. 2003), to analyze data, and became acquainted with some of the uncertainties inherent in the use of models and radar data.
COURSE STRUCTURE EVALUATION: DATA AND METHODS.
The goals of the evaluation were to identify student-reported characteristics of the course that enabled them to comprehend and complete the research project, determine their perceived benefits of the course, and compare those with published outcomes from other undergraduate research experiences. The primary data were pre- and postcourse student interviews, allowing for an in-depth understanding of student perceptions. Interview questions asked students to evaluate the course in terms of how they felt that they learned, such as from whom they gained support during the course and how they thought the course might be improved. Secondary data also included student work and a written class survey at the end of the course. Students were given the option to participate in the evaluative research of the course; all of the 18 students gave consent to be subjects in the course evaluation project, and roughly 20% also agreed to be interviewed to provide additional clarity to their survey responses.
The evaluation methods were guided by the theoretical framework of phenomenology, where the purpose, structure, and universal experience of a phenomenon are determined through participants' perceptions of an event (van Manen 1990). The focus of the evaluation is consistent with the core of phenomenology. The qualitative data were analyzed using the techniques of constant comparative analysis, a process of separating qualitative data (e.g., surveys and interviews) into themes and ideas (Glaser 1965; Strauss and Corbin 1990). The method was repeated with the same data by more than one investigator, to ensure reliability of the themes that were developed (Quardokus 2010). Limitations of this study are the small sample size and the evaluation of a single implementation, but our results also build upon previous evaluations of the CASPiE model (e.g., Wink and Weaver 2008).
COURSE STRUCTURE EVALUATION: RESULTS AND DISCUSSION.
During the qualitative analysis of the interviews, three major themes emerged concerning which experiences students reported as enabling successful completion of the course: the progressive structure of concepts in the course, collaboration with peers, and the motivation to overcome challenges to be a successful part of a larger research effort. It is not surprising that previous atmospheric science education literature has also identified these themes as being successful strategies to include into a course structure [e.g., progressive structure and collaboration (Yarger et al. 2003) and overcoming obstacles to complete authentic research (Gonzalez-Espada and LaDue 2006)].
The first reason for success in the course, as reported by students, was the progressive structure of the course: that is, scaffolding (Table 1: weeks 1–3). This structure was described by one student as, “they do make it so that you understand actually what's going on. . . . Then, we were not just running models and having no idea what we were doing.” She described how this conceptual framework guided her understanding of the structure of the model and its output. To make this structure successful, it was important for the TA to encourage students to actively construct their knowledge during the introductory weeks and to utilize the knowledge later. Otherwise, students would not have been able to work independently later to develop hypotheses and procedures and draw conclusions. An interviewee described the process of developing conceptual models as, “[The TA] . . . separates it out. . . . This is the three things we are doing this week. . . . You are like, ok, this could be a little rough, could be a little hard, but it's not going to be too much.” Support from the TA was a significant reason students noted for their success, highlighting the advantage of TA training in pedagogical techniques.
The second theme emerging from the data was the importance of peer collaboration. Students worked in groups of two or three, but they refer to relationships and learning partnerships that enabled them to share data and draw conclusions across the entire class. The groups relied on other members of the class for support and advice when having difficulties with specific tasks. This learning community developed through the semester and was a necessary component for completing the project. One student described it as, “[The learning is] more like a group learning with each of us having our own little things going on.” Without this learning community, the research conclusions of the module would have been disjointed and may not have accurately integrated the entire class's results.
The third theme in the data was a motivation to be a successful part of a research project.4 Students continually mentioned how important it was to complete the module because it “had to be done”: that is, they knew that the module author's research group was depending on them. Particularly, this became evident from the computer difficulties that occurred during the first module. In one interview, a student described that period as a lot of work but necessary because a graduate student and a larger research group were depending on them. This motivation (which would be absent in a traditional laboratory) was vital to encourage students to dedicate extra time to overcome difficulties.
As students discussed these three themes, they provided material for the second goal of the course evaluation, revealing student outcomes that can be compared to those of other undergraduate research experiences in the literature (e.g., Gonzalez-Espada and LaDue 2006; Seymour et al. 2004). These outcomes included the following: increased confidence in their abilities, the capacity to overcome challenges, and a clarification of career options. The structure of the course was directly related to the former two outcomes. Students reported that the use of conceptual models, as well as the support from the TA and classmates, gave them the confidence to perform the research project and the resiliency to continue even when facing difficulties. The third outcome, a clarification of career choice, was related to the students' new understanding of the process of conducting science. Students explained how performing authentic scientific research gave them a clearer understanding of careers in research. They did not indicate they knew exactly what they wanted for a career or that they had decided to pursue a research career, but rather that they felt more confident in understanding the work of a research scientist.
CONCLUSIONS AND FUTURE WORK.
An authentic research experience can be successfully built into a course and yield benefits noted in other types of research experiences that are usually limited to only a small subset of the students and often later in their academic careers. Research modules assist the current research of the module author and provide students some experience in working with atmospheric data, analysis software, and numerical models. A scaffolding structure of the course is important for successful student outcomes. Activities early in the course develop conceptual models that supply background knowledge and give students an opportunity to develop and refine their understanding and closely align with the activities of the research project. Students found working in teams, as well as their larger learning community, useful for discussing the research goals and results. The importance of their work to the research goals of other groups motivated students to work through setbacks, and the authenticity of the research allowed them to encounter realistic problems, setbacks, and solutions as well as to gain a better understanding of their own interest in pursuing a research career.
Ideally, the sophomore-level laboratory presented here would be the first of a series of research-based laboratories that students would experience in their atmospheric science curriculum, increasing in complexity each year as the students acquire more background knowledge in atmospheric science. The sophomore-level laboratory has now been implemented twice, and a new junior-level laboratory has been implemented once. Future work will continue to evaluate the students' progress in understanding atmospheric science research and the benefits to the students.
ACKNOWLEDGMENTS
The first research module was created by the second author, who used the microphysical model generously supplied by Dr. William A. Cooper. The second module was created by Dr. Jeff Trapp. Both modules were developed in consultation with the coauthors. The module development, assessment, and traveling expenses for student presentations at a professional meeting were supported by NSF Award DUE-0837272.
REFERENCES
Anderson, R. D., 2002: Reforming science teaching: What research says about inquiry. J. Sci. Teach. Educ., 13, 1–12.
Bohren, C. F., 1987: Clouds in a Glass of Beer: Simple Experiments in Atmospheric Physics. John Wiley & Sons, 203 pp.
Cooper, W. A., R. T. Bruinjes, and G. K. Mather, 1997: Calculations pertaining to hygroscopic seeding with flares. J. Appl. Meteor., 36, 1449–1469.
Etherton, B. J., S. C. Arms, L. D. Oolman, G. M. Lackmann, and M. K. Ramamurthy, 2011: Using operational and experimental observations in geoscience education. Bull. Amer. Meteor. Soc., 92, 477–480.
Glaser, B. G., 1965: The constant comparative method of qualitative analysis. Soc. Probl., 12, 436–445.
Gonzalez-Espada, W. J., and D. S. LaDue, 2006: Evaluation of the impact of the NWC REU program compared with other undergraduate research experiences. J. Geosci. Educ., 54, 541–549.
Ivanova, D., 2008: The ERAU Undergraduate Meteorology Program, students' learning, and measures of success. Eos, Trans. Amer. Geophys. Union, 89 (Fall Meeting Suppl.), Abstract ED12A-04.
Lopatto, D., 2004: Survey of Undergraduate Research Experiences (SURE): First findings. CBE Life Sci. Educ., 3, 270–277.
Lopatto, D., 2007: Undergraduate research experiences support science career decisions and active learning. CBE Life Sci. Educ., 6, 297–306.
Lopatto, D., 2009: Science in solution: The impact of undergraduate research on student learning. Research Corporation for Science Advancement Rep., 132 pp. [Available online at http://web.grinnell.edu/sureiii/Science_in_Solution_Lopatto.pdf.]
Lopatto, D., and Coauthors, 2008: Genomics Education Partnership. Science, 322, 684–685.
Morss, R. E., and F. Zhang, 2008: Linking meteorological education to reality: A prototype undergraduate research study of public response to Hurricane Rita forecasts. Bull. Amer. Meteor. Soc., 89, 497–504.
Murray, D., J. McWhirter, S. Wier, and S. Emmerson, 2003: The Integrated Data Viewer—A web-enabled application for scientific analysis and visualization. Preprints, 19th Conf. on IIPS, Long Beach, CA, Amer. Meteor. Soc., 13.2. [Available online at https://ams.confex.com/ams/pdfpapers/57870.pdf.]
Newman, S. B., 2008: A new laboratory exercise to study microscale spatial temperature changes on a university campus. Bull. Amer. Meteor. Soc., 89, 1835–1840.
Quardokus, K. M., 2010: Design and evaluation of an undergraduate laboratory course in atmospheric science research. M.S. thesis, Department of Earth and Atmospheric Sciences, Purdue University, 176 pp.
Russell, C. B., A. K. Bentley, D. J. Wink, and G. C. Weaver, 2010: The Center for Authentic Science Practice in Education: Integrating science research into the undergraduate laboratory curriculum. Making Chemistry Relevant: Strategies for Including All Students in a Learner-Sensitive Classroom Environment, S. Basu-Dutt, Ed., John Wiley & Sons, 193–206.
Rutledge, S. A., P. C. Kennedy, and D. A. Brunkow, 1993: Use of the CSU-CHILL radar in radar meteorology education at Colorado State University. Bull. Amer. Meteor. Soc., 74, 25–32.
Sadler, T. D., S. Burgin, L. McKinney, and L. Ponjuan, 2010: Learning science through research apprenticeships: A critical review of the literature. J. Res. Sci. Teach., 47, 235–256.
Schroeder, J. L., and C. C. Weiss, 2008: Integrating research and education through measurement and analysis. Bull. Amer. Meteor. Soc., 89, 793–798.
Seymour, E., A. Hunter, S. L. Laursen, and T. Deantoni, 2004: Establishing the benefits of research experiences for undergraduates in the sciences: First findings from a three-year study. Sci. Educ., 88, 493–594.
Strauss, A., and J. Corbin, 1990: Basics of Qualitative Research: Grounded Theory Procedures and Techniques. 2nd ed. Sage Publications, 270 pp.
van Manen, M., 1990: Researching Lived Experiences. State University of New York Press, 202 pp.
Windham, T. L., A. J. Stevermer, and R. A. Anthes, 2004: SOARS®: An overview of the program and its first 8 years. Bull. Amer. Meteor. Soc., 85, 43–47.
Wink, D. J., and G. C. Weaver, 2008: Evaluation of the Center for Authentic Science Practice in Education (CASPiE) model of undergraduate research. Workshop Linking Evidence to Promising Practices in STEM Undergraduate Education, Washington, DC, National Research Council, 14 pp.
Wood, D., J. S. Bruner, and G. Ross, 1976: The role of tutoring in problem solving. J. Child Psychol. Psychiatry, 17, 89–100.
Yarger, D. N., R. Thomas, J. P. Boysen, and L. Pease, 2003: Education simulations as learning tools. Bull. Amer. Meteor. Soc., 84, 1489–1490.
In previous years, a laboratory course was not offered at this level. Students were encouraged but not required to take the course. Since the implementation presented here, an upper-level laboratory has been revised to follow a similar format. In that case, the research module is designed to still cover the fundamental content knowledge and skills covered in the lab previously, but lab exercises and problems are replaced with the research tasks for the module, with additional topics covered as the research project dictates.
Such concepts are typically included in an upper-level thermodynamics course during students' junior or senior year.
The laboratory was intended to be taken concurrently with students' first atmospheric science course. Therefore, it was assumed students had very little or no content-specific knowledge.
As is common in both undergraduate research and even some graduate-level research, it is difficult for students early in their careers to grasp the full importance of a research problem, particularly in the short time they are exposed to it in the course. Thus, they appeared to be more motivated by being part of the research effort, entrusting the module author's opinion that the research is indeed important to the field.
