Using Simulations and Screencasts in Online Preclass Activities to Support Student Building of Mental Models

Abstract

As online learning and flipped classes become more important in chemistry instruction, the development of learning materials that can be used to support students’ independent learning of conceptual chemistry content is critical. This paper summarizes the key findings from an eight-year investigation of effective practices for using simulations in preclass introductions to core chemistry concepts with a focus on supporting students’ development of particulate-level models. Student learning gains for six core chemistry concepts were compared for students’ independent use of a simulation using scaffolded instructions versus students’ viewing a screencast of instructors modeling the use of the simulation to answer a series of questions. Though both approaches resulted in student learning gains and provided a solid foundation for subsequent instruction, the screencast approach provided additional benefits. These included avoiding potential simulation limitations and the ability to add instructional content to support student learning. Additionally, studying many iterations of assignments for several different topics yielded an assignment design framework that provides guidelines for instructors looking to create or use simulation-based preclass activities in the classroom to support student learning.

1. Introduction

The development and use of models in science and science instruction are critical in developing conceptual understanding and explanations of the natural world. Such models are particularly important for things that are not readily observable or may be too big, too small, too slow, or too fast to observe [1]. In chemistry, the development of scientifically accurate models for the motion and interactions of particles (atoms, molecules, ions, etc.) is critical to being able to predict and explain physical phenomena [2]. Given that it is not possible to directly observe atoms, ions, or molecules due to their small size, there has been increased development and use of conceptual simulations to support student development of particle-level mental models [3,4,5]. Conceptual simulations, unlike operational simulations that focus on teaching procedural skills, focus on the learning process, and are specifically designed to aid learners in constructing mental models [6]. In chemistry, conceptual simulations allow students to visualize the unobservable and explore how changing variables affect systems. The use of such simulations in science education has been shown to enhance science instruction and improve students’ conceptual understanding and satisfaction [7,8]. When such simulations are used in classroom settings, instructors can provide students with guidance and direct their attention to the more salient aspects of the simulation. However, using simulations in class limits the amount of time students have to process information. As our students enter our classrooms with very different backgrounds, it is reasonable to expect that students may require different amounts of time interacting with a simulation to make sense of it and develop the desired mental models. The benefit of many of these online simulations is that students can access them outside of class where and when they want to and spend as much time as they need engaging with the material [9]. This is one rationale for “flipped” classrooms where students engage in some kind of activity on their own before class (readings, watching videos, etc.) and then spend class time working through problems [10].

The past decade has seen the increased use of the flipped classroom model for STEM instruction in response to the call for more active learning in the classroom [11]. This model frees up in-class time for instructors to support students’ active engagement with tasks designed to reinforce or build upon the foundation of knowledge that students developed in the preclass activity [12]. The flipped classroom has been well studied with several meta-analyses supporting the conclusion that flipped instruction significantly improves student cognitive learning over traditional lecture methods across multiple subject domains [13,14,15,16]. Specifically in chemistry, most of the studies on flipped classrooms at the post-secondary level have focused on introductory and organic chemistry [12]. Further, a systematic review of several of these studies noted wide variations in flipped models and identified key features of flipped models resulting in significant student gains including the following: accountability for completing pre and in-class activities; employing responsive mini-lectures to address common student difficulties; and follow-up post-class practice [12]. Though the ability to increase the amount of active learning is often cited as the key reason for implementing flipped learning, a recent commentary highlights the importance of well-designed preclass activities for promoting conceptual learning [17].

1.1. Effective Preclass Activities

Though preclass activities can take many forms, including textbook reading, lecture videos are the most common forms of preclass activities used in chemistry [12,17]. Studies have shown that textbook readings or video lectures can promote learning gains [18,19], though one study that compared different formats for preclass activities found that video lectures had an advantage over textbook readings [19]. More importantly, studies have shown that students’ active engagement in preclass activities is critical to student learning [12,13,16,17]. This is consistent with the ICAP Framework—Interactive, Constructive, Active, and Passive—which predicts that as students become more engaged with learning materials, their learning will increase [20].

Indeed, studies of flipped models that use video lectures with no methods for assuring students are actively engaged with the materials show small or no student gains [12,13,16]. The ability to embed required questions within a video can help to support student self-assessment and provide accountability for the completion of preclass activities, which may explain why students completing such activities were found to significantly outperform students completing textbook readings [21]. This idea is supported by a recent systematic review of flipped models that identified accountability for the completion of preclass activities as a feature of all flipped models that showed significant increases in course GPA [12]. The importance of the preclass activity is further established by a study comparing a traditional lecture class, with a class that incorporated collaborative in-class active learning activities, and a flipped model with collaborative in-class active learning activities. The results showed that after controlling for prior knowledge, the students in the flipped model significantly outperformed students in the other two classes and that there was no significant difference between the traditional class and the class that just incorporated active learning [18].

Another possible explanation for the positive effects of preclass activities is that they can reduce in-class cognitive load by introducing students to core concepts before discussing them in class. This was the basis of a study by Seery and Donnelly, where they looked at the impacts of preclass videos designed to introduce key concepts in forthcoming lectures that included a quiz to allow students to self-assess their understanding of the material [22]. They found that students with no prior chemistry knowledge improved to the point where there was no difference between their performance on the post-module test and final exam when compared to students who entered the class with prior knowledge. In designing such videos, it was also important for them to consider Meyer’s theory of multimedia learning, which posits that students learn better when the material is explicitly chunked to help organization, only the content relevant to the learning objective is included, the material is presented both visually and verbally, and the means for student self-assessment are included [23].

1.2. Using Simulations in Preclass Activities

In chemistry, the use of conceptual simulations in preclass activities has value in helping students start to develop particle-level mental models for core concepts. This can allow students’ active engagement with the material [24] and the opportunity for discovery learning [25,26,27]. One set of conceptual simulations that has been particularly well studied is the PhET simulations [3,27]. A recent review of 31 experimental or quasi-experimental studies noted that the use of PhET simulations was found to enhance student conceptual understanding of physics when included in inquiry-based activities, virtual labs, problem-based learning activities, and scaffolded learning activities [28]. Further, the use of PhET simulations in introductory concept development activities as part of a learning-cycle-based flipped model in a chemistry course resulted in significant learning gains from pre- to post-assessment and on the final exam for all concepts studied [8]. In this study, authors specifically chose to incorporate these concept development activities in class as opposed to using them as preclass activities as they noted that students typically only have a listening or note-taking role in online learning activities. Another study in physics incorporated PhET simulations into online preclass activities with the goal of enhancing reading assignments [29]. They found that though students found the preclass reading assignments with PhET simulation activities more enjoyable than the reading assignments alone, there was no significant difference in learning gains between the reading-only group and the group that had the reading assignment enhanced with the PhET simulation activity. The authors suggested that one reason for this may have been that did not provide enough scaffolding for students to productively explore the PhET simulation. This is supported by the fact that students reported spending on average about 10 min interacting with the PhET simulations and about 35 min reading.

The need for adequate scaffolding to effectively support student interactions with the simulations is an important consideration, especially as many of the chemistry simulations are quite complex. Though studies of PhET simulations conducted with students in individual interviews or small group settings found that engaged exploration with the simulation only occurred when students were provided with minimal or no guidance [25,30], a subsequent study specifically focused on student use of the PhET simulations individually outside of the classroom found that a higher level of scaffolding was required to meaningfully engage students with the simulations [31].

Even with appropriate scaffolding, especially if the simulations are quite complex, novice learners may not identify important features [14] or may misinterpret some features of the simulation [7,32,33]. Alternatively, a screencast, a screen capture video where an instructor leads students through a simulation, can address some of the disadvantages associated with students’ independent simulation use. With a screencast, instructors can direct students’ attention to key features, thus reducing their cognitive load, and can clarify or prevent misinterpretations of the simulation content or features [34]. However, as students are watching a video as opposed to directly interacting with the simulation, there is the potential for more passive engagement with the content when using screencasts [32].

The goal of the ChemSims Project [35] has been to develop structured support for students’ use of simulations outside of the classroom to help them develop particle-level understanding of core chemistry concepts. To identify effective practices for the development of supporting materials, we compared students’ independent, scaffolded use of simulations with student viewing of screencasts where an instructor demonstrated the use of the simulation to explore a concept. In both cases, students were expected to answer questions while completing the assignment to assess how well they were able to identify key elements of the simulation and apply these observations to the desired core chemistry concept. All activities were designed to be a preclass, initial introduction to a topic that could then provide a foundational experience upon which an instructor could build further understanding. Over the course of the project, materials to support student learning for six different foundational chemistry concepts were developed and studied [36,37,38,39,40]. This paper describes the key takeaways from the ChemSims project regarding the effective practices for using simulations and screencasts to support student learning outside of the classroom and how they can be used in the development of effective preclass activities. Such materials can be used to both support students’ learning outside of the classroom and to support flipped classroom models that provide more in-class time for engaged student learning.

2. ChemSims Assignment and Study Design

In addition to important considerations such as ensuring active engagement of students and providing adequate scaffolding discussed above, the clarity and quality of instruction can certainly play a role in the efficacy of the screencast and simulations assignments. To ensure alignment of learning objectives, assessments, and instruction and provide a mechanism for evaluating areas of student confusion, the screencast and simulation assignments for each of six chemistry concepts were developed and tested using backward design [41] combined with an iterative revision approach (Figure 1), which has been described in more depth elsewhere [36]. Each activity was designed as an introduction to a topic with a specific focus of helping students develop particle-level mental models. Students were given a pretest aligned with the identified learning objectives to establish prior knowledge before engaging with either the simulation or screencast assignment; each assignment was designed to be equivalent in content and focus. Matched posttest questions were embedded in the assignment to allow for a pre–post comparison measure of student learning gains (Figure 2). Qualitative analysis of student written responses to pre and posttest questions as well as questions embedded within the assignment provided data regarding student challenges.

These data were used in the assignment revision process and as evidence for the efficacy of these assignments in supporting student development and use of accurate scientific particulate-level mental models. The revisions were a key aspect of the project and involved refining questions and instruction prompts to address the specific challenges or misinterpretations that the students were experiencing. For all the initial studies the screencasts were designed to parallel the scaffolded use of the simulations. However, for one particularly challenging topic, we incorporated additional instructional elements into the screencast assignment, allowing us to investigate the effects of an “Enhanced Screencast Activity”. These enhancements are discussed in more detail in Section 3.1. Typically, three to four iterations of assignments for each topic were evaluated using mixed methods (Figure 2). Our study of these assignments was guided by two primary research questions:

• How can we use simulations or screencasts to support students’ conceptual understanding in chemistry outside of class?
• What are the benefits and challenges of the guided interactive use of a simulation and the viewing of a screencast of the same topic?

3. Discussion

3.1. Key Takeaways

Based on the combined results of the detailed studies for each individual topic [36,37,38,39,40], we identified several key takeaways for the development of preclass activities that use simulations. They include the following:

• Students were able to learn content effectively from either direct simulation use or from engaging with screencasts paired with answering questions, and the activities raised the average understanding to a similar level regardless of prior knowledge;
• These preclass activities supported student development of particle-level models and provided a common experience that instructors could effectively build upon through classroom instruction;
• Screencasts provided several advantages over student-guided simulation use that included being able to avoid potential simulation limitations or seamlessly adding instructional content to support student learning (Figure 2: Enhanced Screencast);
• Assignment design is effective when following a pattern of orientation, exploration, and application of knowledge and is iteratively revised.

For all the topics we studied, statistical analysis showed significant pre–post gains for both simulation and screencast assignments and in almost all cases, there were no significant differences in the learning gains for students who used the simulation on their own with scaffolded instructions or viewed a screencast where an instructor manipulated the simulation and highlighted key features. In fact, we observed that the class average often rose to a similar level regardless of the starting prior knowledge level (as measured via the pretest question). Figure 3 illustrates this as we see greater variation in the average pretest scores on the (a) Equilibrium and (b) Kinetics assignments than on the posttest score. This suggests the value of using these activities as a way to help mediate differences in students’ incoming background knowledge.

Both styles of preclass activities were effective at helping the students begin to build particle-level mental models. For example, in the Gas Laws pre-assessment, students were asked to use particle motion to explain why a helium-filled weather balloon gets larger as it ascends, but 42% of students provided a macroscopic-level explanation. When asked this same question after completing either the simulation or screencast activity 13.5% (simulation)—30% (screencast) of these students moved to a particle-level explanation [39].

Even though this work demonstrated that either student-guided use of simulations or screencasts can support students’ conceptual learning and building of particle-level mental models, we also identified several advantages in using screencasts that might influence an instructor’s choice. Screencasts can eliminate technology issues that students may encounter when trying to use simulations, for example, using simulations that run on Java, which no longer runs easily on many devices. Additionally, all models and simulations have limitations. Students may hit these limits and obtain “inaccurate” results when they are independently manipulating the variables in a simulation. However, as novice learners students may not recognize these limitations or the inaccuracy of the results leading to incorrect interpretations. For example, when using the PhET reaction and rates simulation [42], students were asked to heat up the system and observe what happened to the total energy. However, many students using the simulation stated that there was no change in the total energy for the system. This was possible because the students had “maxed out” the bar indicating total energy prior to heating the system. Thus, the simulation could not show a change in total energy [38]. This issue was not present for the screencast, as the instructor avoided such a potential source of confusion.

Further, for particularly complex simulations, screencasts allow instructors to better focus students’ attention on key interactions. For example, Figure 4 shows two screens from the water tab of the PhET sugar and salt solutions simulation [43], which was used in our solubility activity. Though the simulation does a good job of illustrating how polar water molecules interact with ions (a) and with the polar sugar molecule (b), as these particles are moving around a lot and there are many of them on the screen, it is challenging for a novice learner to focus in on the specific interactions that best illustrate how water molecules orient themselves around ions and polar molecules. In the assignment, when asked about the interactions between water molecules and sucrose, most screencast students (85%) said that there were interactions between water and sucrose that were similar to those between two water molecules. However, most simulation students (88%) said that there were no interactions or indiscriminate interactions between water and sucrose. This suggests that on their own, even with substantial scaffolding that was revised three times, students were largely unable to discern the interactions between water and sucrose in this complex visual.

It is also generally easier to develop screencasts that highlight the important features, patterns, or interactions in a simulation than it is to provide written scaffolding that will get students to the same place. Though we were often able to find the right scaffolding eventually, it typically required several more iterations of the assignment revision cycle to get the scaffolding in the simulation assignments “right”. This was especially true for complex simulations that have many different parts or variables to pay attention to, and in some cases, we were never able to obtain equal outcomes for simulation and screencast assignments [37,39]. Further, screencasts present material both visually and verbally. Since auditory and visual information is processed through different channels and simultaneously [44], this allows for dual coding, which has been shown to improve learning and retention of material [45,46].

Finally, screencasts allow instructors to supplement the simulations with additional content to further support student learning. For example, in a screencast, an instructor can provide side-by-side pictures of the simulation under separate conditions to better illustrate the effect of changing a variable. Alternatively, an instructor might provide additional images and verbal commentary. We found this to be important for particularly difficult or abstract concepts such as understanding how chemical potential energy is associated with bond breaking and bond forming [36]. Though the Atomic Interactions PhET simulation [47] used to illustrate this relationship is relatively simple with few moving parts, the concept is particularly challenging for students. The simulation focuses on the chemical potential energy changes associated with the formation and breaking of a single attractive force between two particles. Despite multiple iterations of both the simulation and screencast versions of this activity, it was only when we enhanced the screencast (Figure 2: Enhanced Screencast Activity) with additional images and supplemental instruction that explicitly connected the energy changes associated with the breaking and forming of individual bonds to the overall energy changes at the larger system level of a chemical reaction that we saw students better able to make connections between the simulation and the overall exo or endothermic nature of a chemical reaction [36].

3.2. Implications for Assignment Design

Based on these results from the ChemSims project and the use of such simulations and screencast assignments in our classrooms, we have identified three important guidelines in developing preclass activities that incorporate simulations:

• The structure of the assignments should include (a) an orientation, (b) an opportunity for students to identify patterns and make connections, and (c) an opportunity for students to practice and assess their knowledge;
• The activities should be viewed as the starting point of learning, which the instructor can build upon during in-class instruction and student work;
• Employing an online format gives immediate access to student responses that allow the faculty to quickly identify challenges that students are experiencing.

Whether developing a simulation activity or screencast, one goal of the assignment is to help students understand key features of the simulation concerning what variables can be manipulated, how such variables are manipulated, the different types of visuals (e.g., graphs, vectors, particle motion, etc.) available, and how to interpret those visuals. This can be achieved by asking the students directed questions that require them to manipulate variables and make observations (simulation) or systematically demonstrating them (screencast) and asking students to answer related questions. PhET simulations are designed to support student construction of conceptual understanding through exploration with minimal guidance [30,48]. However, our experience was that when used outside the classroom (which lacks immediate instructor support), assignments missing scaffolded orientation would result in students missing critical features of the simulation. This was especially true for more complex simulations with several different variables and display options. After orientation to the simulation, it is important to provide scaffolding that will help students focus their attention on aspects of the simulation to help them identify key interactions or patterns. We found success in having students make a series of related or contrasting observations, sometimes having them summarize their results in a data table, and then asking them to develop conclusions based on the gathered information. For some concepts, it was then possible to have students investigate other relationships on their own. For example, in the gas laws activity, students explored how the pressure of a gas was affected by changes in the volume of the container or type of gas with more guided scaffolding or by watching the screencast and were then asked to determine relationships between other variables (e.g., temperature and pressure) on their own using the simulation. Finally, students’ understanding of key concepts was assessed by asking them to explain the patterns or relationships they identified based on their observations from the simulation and to apply their newly acquired ideas to other relevant phenomena. This allows students to test their knowledge and self-assess their understanding and provides instructors with critical feedback about the level of student knowledge at the end of the activity.

Building scientifically accurate mental models for core science concepts is challenging and takes time and multiple exposures [49]. This is the basis for using these activities as the starting point for student learning. However, the key to building on these activities during in-class instruction is identifying challenges and gaps in student understanding from student responses (Guideline 2). This is made significantly easier when using a platform like Google Forms to collect student responses in an electronic format (Guideline 3). In our experience, quickly scanning through student responses to a particular question can indicate patterns. Example responses that highlight these patterns can be used to drive productive class discussion about the core concept. This can support students in refining and building on their initial ideas while simultaneously validating the time and effort that students put into completing the assignment. Further, it allows instructors an opportunity to normalize making mistakes and refining ideas as an important part of building science knowledge and the learning process in general. Identifying “common responses” given by students and discussing the strengths and weaknesses of each as a class provides important formative feedback to students in a non-threatening manner that does not require an instructor to give individual feedback to each student. We have found that this approach goes a long way toward helping develop a more learner-centered classroom environment and supporting student buy-in for employing preclass activities.

4. Conclusions

A 2012 Report from the National Academies of Sciences focused on the state of discipline-based education research stated that “In general, students have difficulty understanding phenomena and interactions that are not directly observable, including those that involve very large or very small spatial and temporal scales” [1]. Simulations can help students understand these phenomena and interactions and using simulations outside of the classroom can allow students with different incoming background knowledge to engage with the content for as long as they need. The ChemSims project allowed us to explore and evaluate different methods for using simulations to support student building of mental models and conceptual understanding of core chemistry concepts outside of the classroom. Through multiple iterations, we identified an effective activity design strategy of orientation, exploration, and application of knowledge. Findings from this project and previous studies indicate that self-exploration of simulations with appropriate scaffolding can be used to support student learning of core concepts. However, when using simulations outside of the classroom for preclass or homework activities, screencasts may provide several advantages for both students and the instructor. Importantly, screencasts can allow instructors to capitalize on the benefits of dual coding by providing simultaneous visual and auditory information [45] and provide supplemental instruction to extend what can be gleaned from the simulation alone. Further, in a screencast, the instructor can ensure that certain simulation conditions are examined, which may not happen during self-exploration if students are just trying to get through the activity.

5. Future Work

Much research on the flipped learning model has so far focused on the benefits of the increased level of in-class work and collaboration. Yet, recent studies and reviews suggest that the preclass instruction is a critical element of effective flipped models [12,17] and that online options can have some benefits over more traditional reading assignments [19]. One area of future work is to examine what benefits may exist for incorporating these types of particle-level simulations into preclass activities for flipped classrooms. Additionally, though we did not focus on laboratory instruction in this paper, laboratory instruction has been frequently critiqued with respect to its support of student learning [50,51,52]. One reason for this may be the broad set goals for laboratory that range from learning skills to learning concepts. Though conceptual simulations such as the ones discussed in this paper have been used for laboratory investigations, most laboratory simulations are operational, focusing on laboratory procedures. A possible area for future research is looking at the use of conceptual simulations in pre-lab activities to provide a conceptual background. Does having a background understanding of the chemistry concept being studied in a laboratory activity help students get more out of the laboratory activity?

Other future research should focus on using best practices in multimedia learning for the development of preclass activities. As a result of COVID-19, many people gained significant technical savvy in creating online materials, particularly screencasts, so making screencast assignments may not seem like a new concept. However, in most cases, this was done quickly out of immediate necessity and thus, these materials were often not created with best practices in supporting independent student-engaged learning or the principles for effective multimedia learning in mind. Thus, in addition to the considerations we identify above, if planning to develop screencast preclass activities, we strongly recommend exploring Mayer’s 12 Multimedia Principles for Learning [23]. Mayer’s definition of multimedia learning can also be viewed as dual-code or dual-channel learning grounded in dual-coding theory, which posits that the mind processes verbal and visual information through separate channels [45]. Further, he takes the perspective of multimedia learning as knowledge construction. Thus, the goal of multimedia presentations is not just to present information, but also to support the processing of the information by cueing what to pay attention to and how to organize the material, and how to relate the material to prior knowledge. Within this context, several principles stand out as critical in the production of video materials to support student conceptual learning or core chemistry concepts. First, the multimedia principle suggests that people learn best from a combination of words (verbal) and pictures (visual). This means that it is very important to include relevant visuals to support student learning. Second, the coherence principle indicates that learning is more effective when unnecessary information is excluded. This suggests limiting text on the screen to key terms and eliminating any extraneous images or animations that are not core to the concept being discussed, flashy transitions, story-based sidebars, or images of the narrator on the screen during the learning process. These things, as well as background sound or music, which should also be avoided, can distract a learner’s attention away from the key content. A third principle is signaling, which suggests that learning is enhanced when cues such as highlighting, arrows, or circling are used to draw attention to important information. The last highly relevant principle to keep in mind when developing videos is the segmenting principle. Mayer found that learning is more effective when content is broken down into smaller well-articulated units. Features that support this are clear introductions that indicate what will be covered in the video or summaries that summarize what was covered, and title slides or headers that match the wording used in the introduction or summarizing organizers.

Another important consideration in developing screencasts is accessibility. In particular, closed captioning should be accurate, especially for technical words, and contain proper punctuation. Closed captioning is something that many students who do not have auditory challenges use to support their viewing of videos, especially if they use words students are not familiar with. Additionally, if analogies are used in supplementing the content in a simulation to help explain a topic, it is important to ensure all learners can connect to the analogy. One way to achieve this is to provide a visual alongside a verbal explanation that explicitly links the analogy to the science concept.