Ready to learn physics: a team-based learning model for first year university

In $2013$, the School of Chemical and Physical Sciences introduced a brand new topic—'Physics for the Modern World', replacing 'Physics for the Life Sciences'. This topic is also an introductory physics topic and it is accessible to anyone with an interest in understanding the world we live in. It specifically focuses on the ways in which physics underpins our everyday life. Students develop an understanding of natural phenomena through consideration of contemporary issues that humanity faces in the 21st century. They also, regardless of their major or future career plans, see how physics is interesting and relevant in the modern world. The curriculum and teaching methods were designed and trialed to achieve optimum student interest and learning, to provide students with a positive experience, and to bring them closer to being practising scientists by learning the logical approaches physicists use to acquire new knowledge. The topic material contains five units: energy, electromagnetic (EM) waves-environmental effects of EM radiation, global warming and climate change, the nucleus and nuclear energy and cosmology. The four TBL classes associated with these units were: class $1$ = EM waves and environmental effects of EM radiation/global warming, class $2$ = the nucleus, class $3$ = nuclear energy, and class $4$ = energy.

The topic consisted of conventional lectures, practicals and tutorials. Traditionally there were three lectures per week scheduled for the topic, which itself ran for 13 weeks of a semester. We flipped one of the three lectures (the last lecture on Friday) per week and timetabled TBL activities instead. The implementation included four TBL classes which were in a workshop format, which were held in weeks 5, 7, 9, and 11 of the semester, allowing time for preparation activities which were scheduled a week prior to these sessions. Students were informed that this topic would implement TBL, and it was made clear what were the expectations of the class. The TBLs were worth 20% of their final grade, while the weighting for the final exam and the laboratory work was 60% and 20% respectively. Each activity of the TBL carried a percentage of total marks, and each TBL was assessed as 5% for the individual online pre-TBL quiz, 5% for the group quiz and 10% for the in-class group assignment. The topic co-ordinator and lecturer for this topic (first author) randomly formed permanent student groups (groups of 4), with the group listing being made available on our online LMS by week 2 of the semester for the students to access. Each group was given an iconic city name e.g. Paris, so that it was quite easy for the students to remember their group name when they turned up for their class. Through a class-email, we informed the students that their TBL group should be used as a resource for preparing for their TBL class, and for submitting their in-class TBL assignments, and we recommended to them that they stay in the allocated groups for the effective functioning of their team. However, we also gave flexibilty to students to change into a different group for genuine reasons. Nonetheless, we observed that students perferred to stay in their allocated groups.

In phase 1, students needed to engage with the material independently. They were given pre-class preparation materials at least a week before their TBL classes to study. For example, we posted in week 3 some reading material and practice questions on the LMS as part of the pre-class preparation.

In phase 2, students went through a process to to assure their readiness (understanding of basic concepts) to apply their knowledge to the in-class group assignments in phase 3. At the start of phase 2, the students were given an individual RAP test (pre-TBL quiz). For example, by the start of week 4, we posted a multiple choice quiz on the LMS (pre-TBL quiz), which students needed to complete individually, not in a group. The online pre-TBL quiz was created to cover concepts in the form of $10\times 10$ mark questions. The students submitted their answers on the LMS. We set the pre-TBL quiz to allow one attempt and offered randomized variables in questions for each student to solve and hence made them carefully think through the problem.

In week 5, the students had their first TBL class. The lecturer prepared for the class by checking the students' submissions and the whole class performance statistics on the LMS. She then selected the five questions which were challenging (class average of 50% or below for the correct answer) to most of the students in the class. The lecturer then made a power point presentation, with this subset of quiz questions, for the students to retake in the class. During the first 10–15 min of the class, the students needed to complete this subset of the quiz questions, that they had already tried individually on LMS, and together work out the answers. This time, the students did the quiz with their allocated group members. The students interacted with each other to find the correct answer which they recorded in the answer sheet provided. It was pleasing to see 'ah hah' moments during this phase of the process in their learning. When that task was finished the students turned in their group answer sheet to the lecturer. After collecting those answer sheets, the lecturer addressed each of those questions and gave the groups immediate feedback.

Phase 3 used the remaining class time for the groups to complete an in-class assignment which usually consisted of two problems. The in-class assignment is a key component of TBL, so these assignments were designed in such a way as to ensure group interaction and teamwork. Physics is best learned by solving problems, therefore most of the remaining TBL in-class time was spent in workshop format working on problems. At the end of the class, students handed in their group work and received timely feedback. The re-take group quizzes and in-class assignments were marked and distributed in the following week, with written feedback for their in-class assignments. In the week after the TBL class, the students were allowed another individual attempt at the quiz (post-TBL) to see if their knowledge had improved. We offered randomized variables in the quiz questions, for each student to solve, which reduced the chance of them using their pre-TBL quiz answers in the post-TBL quiz. Participation in the post-TBL quiz was not part of their topic assessment as it was primarily conducted for evaluation purposes for this paper. The motivation for the students to sit the quizzes was that similar questions might be used in the final exam and so this was an opportunity for further practice.

To obtain feedback from students, the lecturer began a new thread on the LMS discussion board with the post 'voice your opinion on TBL' and students voluntarily responded.

All students were invited to complete a four-point Likert scale survey on the TBL classes and their preparedness for the classes. We have utilized a slightly modified version of the instrument developed by Vasan et al [19] consisting of 11 statements- the statements are shown in table 1.

To determine whether peer–peer interaction genuinely occured between students in the TBL sessions, the class was recorded using video cameras. For analysis, the video was paused at regular intervals and the group interactions were scored by looking at who was talking and whether or not the other team members were engaged. A camera scanned the class 35 times in a 1 h TBL session. For each video section, if someone was visible, then they were scored two points if they were actively participating with the group (e.g. talking), one point if they were working or listening and zero points if they were not engaged or distracted. These values were averaged across the videos and normalized so that a student who scored one point on every video would have an interaction score of 50%, and a student with two points on every video would have an interaction score of 100% (see equation (1))

$$\begin{eqnarray}&&\mathrm{interaction}\;\mathrm{score}=100\times \displaystyle \frac{(\mathrm{sum}\;\mathrm{of}\;\mathrm{scores}/\mathrm{no}.\mathrm{of}\;\mathrm{videos}\;\mathrm{visible}\;\mathrm{in})}{2}\%.\end{eqnarray} \tag{ 1 }$$

Students were arbitrarily assigned numbers as an identifier and placed into random groups. Each group was then observed and the quality of the student interactions was observed. Higher scores indicate that students were actively participating within the group; while progressively lower scores were given to students if they were working, simply listening or were distracted.

The ability of TBL to promote student learning of key physics concepts was evaluated by experiment, using pre- and post- testing. Elements of the TBL process were assessed as outlined in section 2.1 above. Online quiz questions were developed; some conceptual questions were taken directly from the online instructor question 'banks' and the back of the chapter problems of the textbooks [20–23], and the rest were written by the lecturer (first author). As outlined above, all of the students took this pre-TBL quiz as part of the RAP and it formed part of their assessment. In the week after the TBL class, the students were allowed another attempt at the quiz (post-TBL) to see if their knowledge had improved. Participation in the post-TBL quiz was not part of their topic assessment as it primarly formed part of the evaluation process for this paper and was designed to be formative for the students. The motivation for the students to sit the quizzes was that similar questions might be used in the final exam and so this was an opportunity for further practice. We offered randomized variables in the quiz questions and randomized the question order in order to reduce the chance of students using their pre-TBL quiz answers in the post-TBL quiz. For the post-tests, multiple attempts were permitted to support formative learning but only the first attempt has been analysed. For statistical analysis, we ran a multi-level model [24] with all four physics TBL classes and pre/post results as two separate factors. The multi-level model tested for an interaction between the pre-post scores and the classes i.e. whether or not the change in score was different across classes. The independent variables in the model were class and time (i.e. pre versus post) and interaction between class and time. The student identity code was included as a random effect to account for the correlation in knowledge scores within each student.

This project had ethics approval from Flinders University (project number: 5757 SBREC).

Analysis of student comments on the topic's LMS discussion board revealed four main themes. The students perceived the TBL model to: be enjoyable; help them to learn; provide interaction; and make them spend time on the task. Comments that illustrate these themes are:

Enjoyable

'Really enjoyed TBL, found it quite engaging...' 'I enjoyed it a lot...' 'TBL was fun', 'I enjoyed the TBL's'.

Learning

'Very valuable, I think it will help in my understanding of the broader topic'. 'An effective way to learn'. 'The TBL was a great way to consolidate learning'.

Interaction

'Love the interactive atmosphere, gives a better chance to get to know your fellow peers, engage in learning'. '...loved the interaction...', '...it is very good to discuss problems with the group...', 'It is fantastic to be able to discuss and be able to bounce ideas and answers off of others. It became apparent that (despite feeling it was only myself) several others are also struggling with the same content, we managed to work through some of it together to understand it better'.

Time on task

'Pre-TBL quiz helped (me) prepare for the workshop quiz a lot', 'it is great, good way to work through problems'.

The evaluation of the survey results showed overwhelmingly positive feedback about the TBL activities and how it helped them in their learning process. Of the 51 enrolled students, 36 responded to the survey. Responses to most statements show how TBL helped students in their learning process (table 1 summarizes the responses to the 11 statements). Survey results show that a high propotion of students (97%) felt that TBL helped them to increase their understanding of the topic material. 98% percent of the students strongly agreed (see the largest median score recorded against this response) that solving problems in a group is an effective way to practice what they have learned. Our results also support the findings of previous studies in the literature [12, 25, 26] that students associate increased problem-solving skills with working in groups. Most students indicated that they found working as part of a team, in their classes, a valuable experience. A notable proportion of students (78%) thought that the preparation tests (pre-TBL quizzes) were useful learning activities. Most importantly, the students reported with a very high broad agreement (100%) that they have a positive attitude about working with their peers.

Figure 2 depicts the results of the pre-post tests of all four classes. Only students who participated in both the pre-and post-test were included in the analysis. Of the 51 students enrolled in the topic, the participation rates were N = 29 for class 1, N = 26 for class 2, N = 24 for class 3, and N = 22 for class 4. Given that the pre-test was assessed but the post-test was voluntary these were good return rates. This suggests that students saw a benefit in doing the post-test and maybe in future this should be made available even if it is not part of the assessment.

Zoom In Zoom Out Reset image size

We used the program Stata (StataCorp, USA version 13.0) and the 'mixed' command for mixed models. The graph in figure 2 is a standard box and whiskers plot with the median, interquartile range and whiskers to represent the bulk of the data. There is an overall increase of 1.04 units out of ten from the pre- to post- test results which is significant ($p\lt 0.001$). Note that as these are matched pairs (i.e. the same student pre- and post-), any effect of self-selecting high achieving students is negated by only including in the pre-TBL analysis those who participated in the voluntary post-test. Therefore, TBL produced an overall improvement in performance, equivalent to a 10% mark improvement. Given that there was only one week in between the pre-TBL and post-TBL tests, this impact could reasonably be attributed to the TBL phases 2 and 3 exercises. Interestingly, there was no impact seen in the first class but the median score post-test was better than pre-test in the next three classes. This implies that students may need some time to adjust to TBL before a benefit is seen.

One measure of student engagement is whether or not they stay enrolled in the topic. The dropout rate for first year introductory physics was reduced from 31% in 2012 to 17% in 2013 with the implementation of this model of TBL. Another measure of engagement is whether or not the students actively participated in their teams during the TBL class. Figure 3 shows that most of the students in the study were interacting most of the time. It is interesting to compare the high observed interaction scores (figure 3) with the student responses to the survey question, 'Most students were attentive during TBL workshops', which scored 78% agreement (table 1). Engagement as illustrated by interaction was evidenced by both evaluation methods but the students perceived less interaction than the researchers observed. This could be explained either by the observers not being able to see what kinds of notes etc the students were taking and therefore scoring students more highly; or by students not valuing listening and worked problem solving by team members as participatory activities.

Zoom In Zoom Out Reset image size

For a comparison study, we implemented TBL activities in an environmental science topic which included some physics principles within a geological context. This comparison enabled us to evaluate the transferability of our flipped TBL model to other topics. As described in detail in the methods section of this paper, we flipped one of the three lectures (again, Friday's lecture) per week and timetabled TBL activities instead. Students were invited to complete surveys (the same instrument as described above) that were administered during the last TBL class. For the environmental topic, of the 110 students enrolled 60 responded to the survey, table 2 summarizes the responses to the 11 questions. Responses to most statements are in agreement for both studies and show how TBL helped both groups of students in their learning process. Survey results from both studies show that a high propotion of students (97% for the physics topic and 100% for the environmental topic) felt that TBL helped them to increase their understanding of the topic material. All of the environmental science students (compared to 98% for the physics topic) strongly agreed that solving problems in a group is an effective way to practice what they have learned.

Figure 4 shows a comparison of the results of the pre-post tests for both the physics and environmental Science topics. We can clearly see that after attending the TBL classes, the unadjusted mean scores for both post-tests increased in both cohorts. Moreover, we found a significant improvement ($p\leqslant 0.001$) in the test scores of all the quizzes for the environmental cohort. For the record, 97 students participated in this study (environmental cohort, N = 71; physics cohort N = 26) in week 7, and 81 students (environmental cohort, N = 57; physics cohort N = 24) in week 11. On completion of the post-tests, results were analysed through a paired t-test approach. Analysis of the pre-post testing of the second and fourth TBL quizzes of the physics cohort showed a significant impact ($p\leqslant 0.002$, $p\leqslant 0.001$) on their learning, while there was no notable difference ($p\leqslant 0.05$) in the pre- and post-test of quizzes one and three according to the paired t-test. We found a significant improvement ($p\leqslant 0.001$) in the test scores for all the quizzes for the environmental cohort.

Zoom In Zoom Out Reset image size