Abstract
Gamification has been extensively implemented and studied in corporate settings and has proven to be more effective than traditional employee-training programs, however, few classroom studies of gamification have been reported in the literature. Our study explored the potential of gamified on-line undergraduate physics content as a mechanism to enhance student learning and motivation. Specifically, the main objective of this work was to determine whether extrinsic motivation indicators commonly used in video games could increase student engagement with course content outside of the classroom. Life Science students taking an introductory physics course were provided access to gamified multiple choice quizzes as part of their course assessment. The quizzes incorporated common gaming elements such as points, streaks, leaderboards and achievements, as well as some gamified graphical enhancements and feedback. Student attitudes and performance among those using the gamified quizzes were examined and compared to non-gamified control groups within the same course. Student engagement was quantified through examining student participation above and beyond the minimum course requirements. The results showed that gaming techniques are significantly correlated with increased engagement with course material outside of the classroom. These results may assist instructors in engaging and motivating students outside the classroom through carefully designed online and distance-delivered undergraduate physics content. Furthermore, the gaming elements incorporated in this study were not specifically tied to the physics content and can be easily translated to any educational setting.
Export citation and abstract BibTeX RIS
1. Introduction
The goal of a video-game developer is quite often in line with that of an educational instructor: to take a novice and help them to become an expert. To achieve this, most video games begin with a tutorial before giving the player small, easy challenges to overcome. The player is then introduced to new, more advanced skills throughout the game which are then tested in a series of increasingly difficult challenges to prove mastery before more skills are introduced and the cycle continues. This methodology of guiding a player into mastery is strikingly similar to the educational approach developed by Keller, called personalized system of instruction, in which students set their own pace of education, requiring as many tests as needed to prove mastery [1]. This approach is also similar to that of the well-known online educational platform Khan Academy [2].
Much research exists in the literature that focuses on the effects of violence in video games and video game addiction [3–7], rather than investigating how these disparate arenas can learn from each other. Within the last ten to fifteen years, however, there has been a shift in focus towards understanding motivations of video game players [8, 9], which can have tremendous implications for the world of education. In 2004 there was a special issue of Physics Education devoted to the role that computer games could play in the classroom [10], with investigations into how physics is used in commercially-available games [11–13], as well as concrete examples of how to incorporate game-play in instructional time [14–19]. These articles provide useful guidance on ways to incorporate commercially-available games or individually-developed software in class in order to tap into our students' interests in the gaming culture. Our controlled study was designed to further explore the role that game-based learning could play in a university setting.
In particular, there are a number of techniques that video game developers use to transition a player from a novice to an expert that can be implemented in an educational setting [20, 21]. These techniques include but are not limited to: points, streaks, leaderboards, badges, and instant positive feedback. Points are used to demonstrate to a player that a correct action has been taken, and to provide a player with a sense of improvement as they see their score increase, or when comparing a new score to an old one. Points can be awarded in numerous ways, and may not always follow a linear structure. For example, streaks increase the value of points for performing desired actions consecutively. In rhythm games, streaks are often rewarded in the form of point multipliers for hitting the appropriate note or rhythm many times in a row. Leaderboards allow players to compare their scores and progress to their competitors to place their own score into context. Badges are typically awarded to a player for achieving a goal that is not directly required to complete the game. Badges are often used to demonstrate mastery, as many of the goals that result in a badge are highly difficult tasks, and most players will not receive all the badges in a game. Finally, when a player is successful in a task, they are instantly informed of this, typically through a graphical acknowledgement. In this study, several of these elements found in video games were used to develop a new type of quiz in our introductory physics course, to determine whether these strategies can be used to improve student engagement.
2. The quizzes
There were two styles of quizzes used in this study, the first of which we refer to as the list style. List-style quizzes were the more traditional style of quiz, in which students were presented with a list of multiple-choice questions to answer. Students had access to all the questions simultaneously and were able to answer questions in any order they chose. Students were also able to change their answers as many times as they desired before they submitted their quiz for grading. The grades for list-style quizzes were calculated by the number of marks the student received divided by the total marks available, with 60% being the passing threshold set by course organizers.
We refer to the second style of quiz used in this study as gamified. These quizzes contain the same content as the list style, but were vastly different in delivery. With gamified quizzes, students were given questions one at a time and, upon submitting their answer, they were immediately graded and given the next question. While this removes students' ability to change their responses, it gives them the benefit of receiving immediate feedback [22]. A correct response gave the student points, based upon a formula developed specifically for this project:
Where n is the number of correct responses answered consecutively, N is the total number of correct responses to this point in the quiz, and c and K will be described in detail below. Based on this formula, N increased, and hence the number of points awarded to the student increased, each time the student correctly answered a question. Additionally, the points would scale as the number of consecutive correct responses, n, increased. This awarded more points to students who were able to answer questions consistently.
The variable c was adjusted based on the number of questions in a given quiz, in order to adjust the strength of the consecutive bonus effect. If the strength of the bonus was too large, the gamified quiz did not compare well against the list-style quiz, but if it was too small then the gaming element became negligible. The variable K was adjusted for each quiz to normalize the total number of points so that they were approximately the same across all quizzes during the semester, despite varying quiz lengths. While K could have been adjusted to make the total available points exactly the same across quizzes, it is customary in video games to have scoring systems that end in 0 or 5, so K was adjusted to maintain this practice. In order to pass the quiz, students were required to receive a certain number of points, determined to be approximately of equal difficulty to the list-style quizzes. This was achieved using a combinatorics approach: all the possible scores that may be generated from a gamified quiz were listed from highest to lowest and the number of passing scores was made equal to the number of ways a list-style quiz may be passed. Students taking the gamified quizzes could also earn up to three stars for each quiz depending on the number of points received. Table 1 provides a look at how the point system worked for the first quiz, displaying the various point thresholds, as well as the pass and fail rate near the equivalent list-based quiz threshold.
Table 1. A summary of the properties of a game-based quiz. In the list-style quiz, a pass is granted when 60% of the questions are answered correctly, or 12 out of 19 questions.
| Property | Value |
|---|---|
| Total number of questions | 19 |
| c, K used in points formula | 10, 20 |
| Points needed to pass | 1656 |
| Failure rate among students getting a total of 12 of 19 questions correct | 6% |
| Passing rate among students getting a total of 11 of 19 questions correct | 4% |
| Points needed for: | |
| (a) One star | (a) 2382 |
| (b) Two stars | (b) 4402 |
| (c) Three stars | (c) 8360 (all questions answered correctly) |
In the second iteration of the gamified quiz, additional gamified elements were added. These modifications included a leaderboard, an achievement system, gamified graphical updates, and a more detailed test summary. The leaderboard allowed students to choose to display their scores for other students to see. The achievement system gave students an extra motivational tool allowing graphical acknowledgement (badges) for accomplishing certain tasks, such as accumulating three stars in every quiz or completing all the quizzes within a stipulated time frame. The graphical update introduced a progress bar during quizzes that updated as students received points, allowing them to see their progress in passing the quiz and achieving stars. Finally, the detailed test summary provided feedback by allowing students to compare their response to the correct response. The detailed test summary was added to both the gamified and the list-style quizzes.
Overall, the gamified features used in this study were bonus streaks, points, instant feedback, achievements/badges, leaderboards, and stars. Table 2 shows a summary of the differences between the gamified and list-style quizzes.
Table 2. A summary of the differences between the quiz styles used in this study.
| List-style | Gamified |
|---|---|
| All questions available simultaneously | Questions are given one question at a time |
| A correct response gives a student a mark | A correct response gives a student points |
| The value of a question is fixed | The points received for a correct response depends on streaks |
| Final grade presented as | Final grade presented as |
| Number of correct responses/total number of questions | Total score |
| A passing mark is 60% | A passing score is set to be of equivalent difficulty to 60% |
The quizzes covered a variety of physics concepts from the course including traveling and standing waves, acoustics, optics, diffraction, and the absorption and emission of light by molecules. Additional questions tested basic mathematical concepts such as trigonometry ratios and logarithms and exponents. Table 3 provides a sample of some of the questions that were asked throughout the quizzes.
Table 3. A sample of questions found in the quizzes.
| What is the intensity level (IL) for a sound of intensity 10−12 W m−2? | ||||
| 10−12 db | 20 db | 10 db | 0 db | |
| One of the statements below is incorrect. Which one? | ||||
| Snell's law is: n1sinθ1 = n2sinθ2 | The angle of reflection = the angle of incidence | The angle of reflection = the angle of refraction | n = c/v, where n is the index of refraction and v is the speed of light in the medium, and c is the speed of light in a vacuum. | |
| Given below are several equations concerning the dynamics of particles. In these equations p is the momentum, m is the particle mass, v is the particle speed (non-relativistic), K is the particle kinetic energy, λ is the particle de Broglie wavelength, and h is Planck's constant. One of the equations is incorrect; which one? | ||||
| p = mv | K = (1/2)mv2 | K = p2/2m | p = h/λ | λ = h(2mK)1/2 |
3. Methods
3.1. Participants and procedure
Students in a first year undergraduate Physics for Life Sciences class completed a series of four multiple-choice quizzes throughout the term. The quizzes did not count directly towards the student's final grade, but students were required to achieve a minimum of 60% on the quiz in order to gain access to other course assessments that were worth a total of 40% of their final grade. Students were divided into two groups, with each group required to use either the list-style or gamified quizzes, in order to compare the two approaches. This study was repeated over two semesters: in the fall 2014 term, students were divided into the two different groups based on class section, while the winter 2015 term randomized students throughout the two sections. The results presented here are the combined data from two semesters in order to increase the sample size of the overall study. The total number of students who completed list-style or gamified quizzes in this study is shown in figure 1.
Figure 1. Participation in the study by quiz style.
Download figure:
Standard image High-resolution image3.2. Measures
Every time a student clicked the mouse on the interface, a record of what they clicked and the date and time of the click was recorded. Additionally the correct responses for each marked quiz were recorded, as well as the marks or points associated with each question. Finally, the total score for each quiz a student attempted was recorded.
The first measurement of engagement was a comparison of the number of attempts students made after reaching the passing threshold for their quizzes. Since students were required to reach the passing threshold in order to access graded course material, all students were motivated to do well on the quizzes initially. However, once a student reached the passing threshold for the quiz, no additional motivation was provided to them in the context of course work. While students were able to retake the quiz as many times as they liked, a higher mark on the quiz did not directly benefit their final grade in any way. Hence, any additional attempts made by a student after reaching the passing threshold of the quiz could be used as a measure of the student's level of engagement. The average number of additional attempts made by students in each group was compared. Since the majority of students were expected to not make any additional attempts, the sample distribution would not be normalized and so a Mann–Whitney statistical test was used to test for significance.
The second measure of engagement investigated was the percentage of students who made repeated attempts, after reaching the passing threshold, in order to achieve a perfect score. As stated before, all students were motivated to do well on their quiz in order to achieve a passing grade. In attempting to pass the threshold, a student may have achieved a perfect score on their quiz. However, if a student who passed their quiz without a perfect score were to then return to the quiz and continue to make attempts until a perfect score was achieved, increased engagement would be demonstrated. Hence, the percentage of students who continued to make attempts until a perfect score was achieved was compared between the groups. Since this measure of engagement only looked at the proportion of students in each group who completed this condition, a Pearson Chi-Squared test was used to test for statistical significance.
4. Results
The number of attempts made after a student had already achieved the passing threshold was compared between the two groups for each quiz, as shown in figure 2. The average number of attempts after passing is significantly greater in the gamified group compared to the list-style group by a factor of 2.8 (p = 0.124), 1.8 (p = 0.045), 3.4 (p = 0.024), and 6.3 (p = 0.015) for the four quizzes respectively.
Figure 2. Average number of attempts after achieving passing threshold.
Download figure:
Standard image High-resolution imageFigure 3 demonstrates the proportion of students who continued attempting the quiz after achieving the passing threshold, until they earned a perfect score. A direct comparison can be seen in figure 3 between those with the gamified and those with the list-style quizzes. The percentage of students who met this condition increased from the list-style to the gamified groups for the four quizzes by a factor of 6.6 (p = 0.027), 1.7 (p = 0.144), 13 (p = 0.001) and 10 (p = 0.004) respectively. This shows that a significantly higher percentage of students in the gamified quiz group consistently worked towards a perfect score after achieving the passing threshold compared with their counterparts in the list-style group. This demonstrates that students who were in the gamified group were significantly more engaged in this course component, and perhaps had a high level of motivation to achieve a perfect score.
Figure 3. Percentage of students who continued taking the quiz after reaching the passing threshold until achieving a perfect score.
Download figure:
Standard image High-resolution imageThe number of attempts that were required before passing the quiz was also examined. As seen in figure 4, students in the list-style group required, on average, fewer attempts in order to pass a quiz than those in the gamified group. While this trend is apparent for all four quizzes, the difference is only statistically significant for the third and fourth quizzes (p = 0.739, 0.618, 0.000, 0.011, respectively).
Figure 4. Average number of attempts to achieve passing threshold.
Download figure:
Standard image High-resolution image
5. Discussion
There are several indications to suggest that the gamified quizzes provided students with a higher level of engagement compared to the control group. Since students only required a passing grade, and the actual score on the quiz was not used directly to determine the students' final grades, there was little incentive for students to attempt to improve upon their quiz scores once they had surpassed the required threshold. The gamified quizzes did not contain any additional features for students compared to the list-style quizzes, such as bonus grades or extra content. Despite this, the gamified group made significantly more attempts overall in every quiz. In particular, students in this group were more likely to make additional attempts after passing their quiz. This demonstrated that students were motivated to use the gamified quizzes beyond their intended role in the overall course structure.
What was even more enlightening was the percentage of students in the gamified group who continued to make attempts after reaching the passing threshold until a perfect score was achieved. Unlike the list-style quizzes, students in the gamified group would receive three stars for a perfect quiz score, and this would be displayed on their home screen profile. The inclusion of the performance-based stars appears to have had a significant influence on these students; on average, over 10% of students in the gamified group demonstrated additional effort to achieve a perfect score, despite not being a course requirement. This is a significantly higher percentage of students than found with the list-style group. This provides support for the hypothesis that gamification can enhance engagement of students with course material.
Students in the list-style group required, on average, fewer attempts to pass the quiz than students in the gamified group, suggesting that the gamified quiz may have been more difficult. Since the knowledge content was identical in the two quiz types, this discrepancy may be explained as a result of small logistical advantages the list-style group had over the gamified group. For example, students taking list-style quizzes had more time to reconsider earlier questions as their answers were not finalized until the entire quiz was done, while gamified quizzes required students to finalize their answer after every question. In addition, students were able to see multiple questions simultaneously, which gave students an opportunity to deduce patterns and use a problem-solving approach to find solutions to groups of questions within the same topic with the list-style quizzes.
Engagement has often been studied under the proxy of on-task behaviour [23]. In this way, various techniques have been attempted in order to improve the time students spend on task [14]. Although the term on-task has been used somewhat inconsistently, there has been significant research demonstrating the importance of time on-task with respect to improved academic outcomes, provided that the tasks are carefully designed and implemented [24, 25]. In our study, we have demonstrated that students who participate in gamification complete more quiz attempts and make greater efforts to achieve perfect than those in control groups, resulting in greater time on-task. While the task given to students in this study was not representative of the difficulty of the rest of the course due to its role in the course structure, the prior literature implies that if the task were to be more representative of the material, students' academic achievements would improve with the increased time on-task observed with gamified quizzes. This is the aim of our further investigations on gamification in undergraduate physics education.
6. Conclusion
Students were more engaged when gamified techniques were used in their quizzes as indicated by more attempts made per quiz and a greater percentage of students striving for perfect scores than seen in the control group. Students who are more engaged and who seek out perfect scores are more likely to look through their notes to find the necessary information to answer the quiz questions correctly. Additionally, the bonus streaks encourage consistency from students who wish to avoid ruining the streak. This may encourage more thoughtfulness from the students resulting in less guessing and, once again, more referring to notes. In this way, gamified quizzes have demonstrated exciting potential to improve learning outcomes in students.
Acknowledgments
The authors are pleased to acknowledge Boom Digital Media Group for the development of The GOPE, as well as assistance from Cindy Wells, Bill Teesdale, James Howard and Chris Schultz-Nielsen.