The effect of using Kahoot! for learning – A literature review

Kahoot! is a game-based learning platform used to review students ’ knowledge, for formative assessment or as a break from traditional classroom activities. It is among the most popular game-based learning platforms, with 70 million monthly active unique users and used by 50% of US K-12 students. Since the platform was released in 2013, many studies have been published on the effect of using Kahoot! in the classroom


Introduction
Most teachers acknowledge that it is a challenge to keep the students' motivation, engagement, and concentration over time in a lecture. Lack of motivation can result in a reduction of learning outcomes and a negative atmosphere in the classroom (O. L. Liu, Bridgeman, & Adler, 2012). This challenge is usually even a bigger problem in higher education with big classes with little interaction. Educational research has shown that students who are actively involved in the learning activity will learn more than passive students (Butler, 1992;Murray, 1991). Further, there is extensive evidence that student engagement in lectures improves understanding and academic results (Prince, 2004). There are multiple approaches for making lectures more interactive, including breaking the class into smaller groups, questioning the audience, using audience responses (systems), introduce cases the students can work on, use written material, organizing debates, reaction panels and guest talks, using simulations and role-plays, using video, audiovisual aids, and using effective presentation skills (Steinert & Snell, 1999). Student response systems (SRSs) were developed in the sixties as a solution to make large classes more interactive (SRSs) (Judson, 2002), and SRSs have been used in classrooms since the early seventies (Bessler & have included game-features as a part of their platform. One platform sharing many of the same characteristics with Kahoot! is Socrative (Coca & Slisko, 2013). Socrative provides a real-time formative assessment to collect data from the students through forms and offers the game Space Race, where teams of students answer questions to move their rocket as fast as possible across the screen. Another example is Quizlet, where students can study various topics through Flashcards, a speller, tests, and a Space Race game where the player can kill moving terms by answering the correct word (Gruenstein, McGraw, & Sutherland, 2009). Quizlet focuses on spelling words and giving the correct definition for words. Quizizz is game-based learning platform similar to Kahoot! where the main difference is that both the questions and the answers are shown on the student devices, it is not necessary to use a projected screen, and answering sessions are not synchronized meaning that a student does not have to wait for other fellow students before continuing to the next question (Chaiyo & Nokham, 2017). Poll Everywhere is an SRS for collecting audience responses in real-time to multiple-choice or open questions (Shon & Smith, 2011) that recently have added game features similar to Kahoot! through Poll Everywhere Competitions. There are also several SRSs available that does not offer game features such as Learning Catalytics which makes it possible for students to give numerical, algebraic, textual or graphical responses (Schell, Lukoff, & Mazur, 2013); iClicker that can be integrated with learning management systems and presentation tools (such as PowerPoint) and where the students both can respond using specialized iClicker remotes or web-based clients (Lucas, 2009); and Plicker where the students give their responses using Plicker cards with a unique pattern for each student that can be rotated to give four different responses recognized by a camera on the teacher's digital device (Krause, O'Neil, & Dauenhauer, 2017). The main difference between all the systems mentioned above is that Kahoot! focuses more on engagement through a competitive gaming experience.
In the search for literature, four literature reviews where Kahoot! was mentioned were found. In a literature review researching trends in student response systems, the benefits of SRSs were summarized to be to provide interactivity, improve academic performance, and engagement, while the challenges are waiting time, academic inefficacy, and practical drawbacks (Aljaloud, Gromik, Billingsley, & Kwan, 2015). Kahoot! is mentioned related to the future of SRS, where it states: "On the other hand, some SRS applications such as Kahoot! have synthesized the best aspects of SRS and smartphone applications by introducing a competitive game element to SRS" (Aljaloud et al., 2015). Another literature review examined the benefits and limitations related to computer game-based foreign language learning, where it was found that this approach seems to be especially effective in vocabulary acquisition (Klimova & Kacetl, 2018). The advantages were higher motivation and increased engagement. At the same time, the disadvantages were a lack of students' concentration on vocabulary acquisition and learning, inappropriate choice of games not relevant, and unfamiliarity of computer games among teachers and their unwillingness and anxiety to use them. Kahoot! was mentioned as a game-based platform useful for foreign language learning.
Further, one literature review looked at online formative assessments and focused on diverse delivery methods and psychological benefits (McLaughlin & Yan, 2017). In this study, Kahoot! was described as a game-like student response system more dynamic than some other tools with the support for video, pictures, music, scoring, and ranking. The benefits of using online formative assessment tools include gains in achievement scores, and the development of essential complex cognitive processes, such as self-regulation. Finally, one literature review studied publications from 2009 to 2018 on mobile-based assessments (Nikou & Economides, 2018). The findings in this article include that most of the reviewed articles reported a significant positive impact on student learning performance, motivation, and attitude and that more research is needed to investigate issues and concerns related to negative perceptions against mobile assessment, especially from the teachers' point of view. One study on Kahoot! was included in this review, which investigated the effect of frequent use of a GSRS over time (Wang, 2015). This review has a similar theme as our review, focusing on the impact of mobile-based assessment on learning, motivation, and students' and teachers' attitudes and perceptions. The main difference that (Nikou & Economides, 2018) focuses on mobile-based assessment in general and our review only on the game-based mobile-based assessment platform Kahoot! and that their study included only a limited number of major referred ed-tech journals.

Research goal, questions and approach
The research goal of our literature review was to investigate research studies on the effects of using Kahoot! for learning. Specifically, we wanted to investigate how Kahoot! affects learning performance, classroom dynamics, student anxiety, and the perceptions of students and teachers. The research method used is based on the Goal, Question, Metrics (GQM) approach (Basili, 1992). First, a research goal (conceptual level) is defined, then a set of research questions (operational level), and finally, a set of metrics to answer the research questions (quantitative level) is described. The GQM approach was chosen, as it provides a framework for transitioning from a research goal to research questions and metrics, and it provides a framework for presenting the research results. In our case, the metrics used to give answers is data extracted from the literature review.
The research goal was defined according to the GQL template (Basili, 1992): The purpose of this study is to investigate the effect of using Kahoot! in learning from the point of view of a researcher in the context of students and teachers in education.
The research questions (RQs) were defined by a combination of decomposing the research goal, an initial search on the most cited studies on Kahoot! information acquired from presentations and talks at learning conferences (such as SXSW EDU, ISTE, and CUE), and enquires of Kahoot! employees. The initial literature search showed that many articles on Kahoot! have focus on the learning effects (Hsiu-Ting Hung, 2017aIwamoto, Hargis, Taitano, & Vuong, 2017;Wang, Zhu, & Saetre, 2016;Wichadee & Pattanapichet, 2018), on how students and teachers perceive the game-based learning platform (Bicen & Kocakoyun, 2018;Chaiyo & Nokham, 2017;Plump & LaRosa, 2017;Wang, 2015), and on classroom dynamics (Cutri, Marim, Cordeiro, Gil, & Guerald, 2016;Licorish, Owen, Daniel, & George, 2018;Wang & Lieberoth, 2016). Some teachers are afraid that the competitive gaming elements in Kahoot! will create anxiety for the students that will ruin their motivation to learn (Zarzycka-Piskorz, 2016). This issue has been identified by Kahoot! staff as one of the main barriers for many to start using the tool. The following research questions were defined: -RQ1: How does Kahoot! affect learning performance? -RQ2: How does Kahoot! affect classroom dynamics? -RQ3: How does Kahoot! affect students' anxiety? -RQ4: What are the students' perceptions of Kahoot!? (related to motivation, concentration, enjoyment, perceived learning and similar) -RQ5: What are the teachers' perceptions of Kahoot!?

Research methodology for the literature review
This study can be classified as a literature review where the search was comprehensive, the appraisal did not include quality assessment, the synthesis was narrative, and the analysis was thematic (Grant & Booth, 2009). The study was carried out according to (Dybå & Dingsøyr, 2008) consisting of five stages: 1) Development of review protocol, 2) Identification of inclusion and exclusion criteria, 3) Search for relevant studies, 4) Critical appraisal, 5) Data extraction, and 6) Synthesis. The following sections will describe each stage. Fig. 1 shows the PRISMA flow diagram for the literature review that gives an overview of the process going from the initial search of the literature (1232 articles), screening, critical appraisal, and to the resulting articles in the qualitative and quantitative analysis.

Development of review protocol
The review protocol developed to achieve the following goals: 1) to maximize the literature coverage; 2) to identify and include the related work that can be classified as a study (experiments, surveys, case studies or similar): and 3) to collect and synthesize meaningful data from the sources related to the defined research questions (see Section 2.3). This protocol specified the research questions, search strategy, inclusion, exclusion and quality criteria, data extraction, and methods of synthesis.

Identification of inclusion and exclusion criteria
The identification of inclusion and exclusion criteria was optimized for identifying as many relevant articles as possible. The inclusion and exclusion criteria are template criteria used by several researchers at our university department. However, one criterion referring specifically to Kahoot! was tailored to this particular study by the authors. A more elaborate inclusion and exclusion were to be carried out in the critical appraisal step. The inclusion criteria used in the search for relevant studies were: � The article is an article and not a report, book chapter, letter, or abstract. � The article is published in an international peer-reviewed journal or conference. � The article is written in English. � The article refers to Kahoot! in the title or abstract.
The exclusion criteria were: � The article is not accessible through university services or memberships. � The article is only accessible behind a paywall. � Kahoot! is only mentioned as an example and is not the focus of the paper.

Search for relevant studies
The search for relevant studies was carried out in two steps: 1) digital research databases were searched for relevant studies, and 2) references in found studies were checked for additional studies. The search string for this review was "Kahoot", and the following five research databases were searched in sequence: Google Scholar, Science Direct, Wiley InterScience, Web of Science, and Scopus. The search was done for articles written in English and for articles published in proceedings or journals. Science Direct, Web of Science, Scopus, and Wiley InterScience were chosen because of their ranking as academic research databases, good coverage of studies relevant for our review, and based and previous experience. Google Scholar was chosen because of its high coverage. The disadvantages of using Google Scholar were the lack of support for specifying article types and articles in other languages with autotranslated titles and abstracts in English. Table 1 shows the results of the search for articles in the five research databases.
At this stage, the title and the abstract of the articles were checked. If an article fulfilled the inclusion and exclusion criteria, the pdf and site data were downloaded, and citation data and keywords were added to a spreadsheet. A total of 206 qualified articles were accepted at this stage, where 187 came from Google Scholar, and 19 from the four other research databases. Most rejected articles at this stage were not accessible through university services or memberships, were project descriptions or abstracts, or articles in another language with English title and abstract.

Critical appraisal
The focus of the critical appraisal was on relevance (only articles classified as an experiment, survey case study or similar with focus on Kahoot!), rigor (appropriate research approach described including research context, number of subjects, scope, design, methods, and execution), and credibility (conclusions based on sound analysis and reasoning). The critical appraisal was carried out by two researchers who independently evaluated which studies to be accepted and rejected. The two lists of rejected candidates were compared and merged into one list. There was only a disagreement regarding two articles, whom both researchers agree to reject but for slightly different reasons. Studies with few subjects (N) were accepted only if they included qualitative data and analysis with sufficient depth. Table 2 shows the results from the critical appraisal step.
Ninety-three of the 206 articles were accepted. Thirty-three studies were excluded as these were studies where Kahoot! was used together with other tools without sufficient description of Kahoot!'s contribution. For the remaining, fourteen were classified as presentations, twenty-two were reviews or comparisons, and forty-four lacked sufficient rigor (see above). A complete overview of the accepted and rejected articles with reasons for why they were rejected can be found at https://bit.ly/2ljrh5l. Of the 133 rejected articles, four reported mainly negative results and the remaining neutral or positive results.

Data extraction
During this stage, data were extracted from the 93 accepted articles by reading through the whole articles in detail. The following data was entered into a spreadsheet: Type of article, the number of subjects in the study (N), a short description of the study, a description of the results, the theme of the article, the context of the study, and a summary of the article. Also, the main contribution in the articles was highlighted in the PDFs to make it quicker to check later for details in the articles.

Synthesis
For the synthesis step, all articles in the review were classified according to the type of study and by nine attributes, as presented in Table 3. The first five attributes represent the five research questions presented in Section 2.3. The purpose of the attributes was to classify the articles according to the five research questions and to what kind of results they provided. The first five attributes were coded by studying the results found in the articles. Similarly, the four remaining attributes were captured by studying the research design, analysis, and results. For each article, all attributes that matched the study were ticked off, and the related text was added to the results and description of the study in the spreadsheet. Then all this information was collected and categorized. The results from all the accepted studies were summarized according to the five research questions and the kind of results reported.
The process of searching for relevant studies, filtering, data extraction, and synthesis took place from February 4th to May 15th, 2019. A secondary synthesis adding effect size and classification according to quantitative and qualitative data were added from September 1st to 20th, 2019.

Results
This section describes the general results from the literature review and the results organized according to the five research questions, as presented in Section 2.3.  Table 4 in Appendix A presents the results of the synthesis of the accepted studies. The table lists the title of the article, the type of study, the nine attributes (see Section 2.4.6), the number of subjects (N), and a citation. Fig. 3 shows the percentage of articles that can be described according to the same nine attributes. The figure reveals that the majority of studies (88%) have a focus on how students perceive the use of Kahoot! for learning and that many studies focus on actual learning (39%) and classroom dynamics (35%). The topics covered by the least number of articles are student anxiety and teacher perceptions. Of the accepted studies, 38% can be classified as experiments, 45% include testing of statistical significance, and 10% include analysis of effect sizes. The effect sizes of an additional 4% of the studies were computed based on the data provided in the articles. Further, 86% of the studies contain an analysis of quantitative data, and about half (52%) contains an analysis of qualitative data.

General results
There is a considerable variation in the number of participants (N) in the studies, where the smallest study only had 5 participants (qualitative study) while the largest had 1000. The average number of participants is 105. Eighty-four percent of the studies were  carried out in the context of higher education and 16% in K-12. For studies in K-12, half of them were in High Schools, and the remaining were equally divided into studies in Primary and Secondary Schools. We did not find any studies of using Kahoot! in other contexts, like professional business training. The total number of citations of the accepted articles was 1042, and the average per article was 11.20. One-fifth of the articles had 0 citations, as over half of the articles were published in 2018 and 2019. The most cited articles are (Wang, 2015) with 238 citations (Plump & LaRosa, 2017), with 87 citations (Wang & Lieberoth, 2016), with 73 citations, and (Zarzycka-Piskorz, 2016) with 68.

RQ1: Learning effect of using kahoot!
This section presents studies related to the actual learning effect from using Kahoot! and how the use of Kahoot! affects learning compared to other tools and approaches.
In this literature review, we found in total forty-eight studies that investigated the learning outcome of using Kahoot! in education (not counting the excluded papers). Twelve of these studies compared traditional teaching to Kahoot! used in combination with other tools in the context of flipped classrooms, game-based learning, blended learning, and technology-supported learning. All these studies show improved academic results for the non-traditional teaching approaches where Kahoot! was used as one of the tools. However, these studies do not explicitly describe how Kahoot! contributed to improving the learning outcome. One example is a quasiexperiment where the students were exposed to technology-supported learning using PowToon, Blendspace, and Kahoot! performed significantly better on average test scores in the course (p < .02) than for those who had traditional teaching (Sarkar, Ford, & Manzo, 2017). Similarly, a quasi-experiment in a programming course investigated the effect of giving traditional teaching vs. giving gamified teaching using Who-Wants-To-Become-A-Millionaire, Kahoot! and Codeacademy programming (Fotaris, Mastoras,  Leinfellner, & Rosunally, 2016). The results showed many positive effects for the gamified approach, including improved class attendance, less late arrivals to class, higher downloads of course material, improved classroom dynamics, and higher final grade (61% for gamified vs. 53% for traditional). Another example is a quasi-experiment where one group received traditional teaching using paper, while the other received technology-supported learning using Voki, Buncee, Kahoot! and Answergarden (Filologiczna, 2016). Both groups performed the same on the pre-test, but the technology-supported learning group performed significantly better on the post-test (scored 90%) compared to those who used paper (scored 75%).
Of the thirty-six studies investigating the learning outcome of only using Kahoot! for teaching, six are in the K-12 context (Primary, Secondary, and High School), while the rest are with university students. The studies are both qualitative and quantitative, where the majority can be classified as quasi-experiments. These studies covers a wide range of fields and courses including language (English and Chinese), Information Technology, Bio-Engineering, Media and Communication, Electric Circuits, Business, Math, Physics, Chemistry, Animal Science, Academic Writing, Educational Technology, Nursing, Vocational Training, Programming, Control systems, and Earth Science. Seventy percent of studies with statistical significance tests on learning effect show that Kahoot! significantly improves the final grade or test results compared to other teaching approaches. However, there are some exceptions. In an experiment in an Information Technology course, Kahoot! did not result in a significantly improved learning effect compared to using a paper quiz and the Clicker student response system . Similar results were found in a quasi-experiment comparing performance of Kahoot! and SurveyMonkey (Tan & Saucerman, 2017), in an experiment comparing PowerPoint and Kahoot! (Stoyanova, Tuparova, & Samardzhiev, 2017), a quasi-experiment in a Middle School investigating the effect of Kahoot! vs. traditional teaching (Lee, Hao, Lee, Sim, & Huang, 2019), and in an experiment comparing traditional teaching with Quizizz and Kahoot! (three experimental groups) (G€ oksün & Gürsoy, 2019). In one study where Kahoot! was used for teaching Chinese, using Kahoot! had a significant improvement in the competences of reading and speaking, but not for listening nor vocabulary (X. Liu & Wang, 2017).
We found twenty studies with experiments that showed a statistically significant increase in learning from using Kahoot! in classroom teaching. Three of these studies focused on how the use of Kahoot! in flipped classrooms improved learning. The post-test in quasi-experiment in Taiwan with 44 students in an English course showed that the experimental group using Kahoot! had an average score of 86.18 compared to 77.45 for the control group not using Kahoot! (p ¼ .007, effect size large) (Hsiu-Ting Hung, 2017aHung, , 2017b. Similarly, another quasi-experiment on using Kahoot! in a flipped classroom showed that more elaborate use of Kahoot! improved speaking skills significantly (p ¼ .010, effect size large) (Hsiu-Ting Hung, 2017aHung, , 2017b). An Austrian study of a C-programming course where 60 students participated showed that the grades improved 12% when using Kahoot! in the context of a flipped classroom (Dolezal, Posekany, Motschnig, Kirchweger, & Pucher, 2018).
Sixty-four percent of the studies on learning effect from Kahoot! in our review describes experiments where the effect of Kahoot! was compared to traditional teaching. In a quasi-experiment in Spain with 89 chemistry students, a control group exposed to traditional teaching was compared to two groups using Kahoot! where one was using Kahoot! twice as much the other (Ares, Bernal, Nozal, S� anchez, & Bernal, 2018). The results showed a significant improvement in the final grade from traditional teaching to the use of Kahoot! (both groups), and that the group using Kahoot! frequently improved more (þ38.4%) than the other (þ32.0%). Further, 20% more students passed the exam with less frequent use of Kahoot! than traditional teaching and 29% with frequent use. A quasi-experiment with 400 students in an Educational technology course at the University of Florence compared traditional with game-based teaching using Kahoot! through a pre-and post-test on theoretical and practical topics (Ranieri, Raffaghelli, & Bruni, 2018). The results showed that the use of Kahoot! had a significant improvement from pre-to post-test for all topics (p ¼ .001, effect size large), but compared to the traditional approach, the Kahoot! group, performed only significantly better on theoretical topics (p ¼ .0001, effect size small) and not on the more practical topics.
An experiment with 96 students at Purdue University in the USA where Kahoot! was used over seven weeks, found that the experimental group did significantly better on the final exam (79.56 for the experimental group vs. 56.83 for the control group) (p < .0001, effect size large) (Bawa, 2018). Similarly, the results of a quasi-experiment with 67 university students in Greece learning electric circuits showed that the experimental group scored 59.93% vs. 51.72% for the control group (p ¼ .001) (Tsihouridis, Vavougios, & Ioannidis, 2017). A quasi-experiment among 98 nursing students at La Salle University examined how including four 20 min Kahoot! sessions would affect the final examination scores (Kinder & Kurz, 2018). The results showed that including these Kahoot! sessions had a significant impact on the final examination scores (p ¼ .005). A case study from Portugal with 324 university students showed that with Kahoot! the average grade was improved 6.4% compared to traditional teaching, that weaker students improved their grades with 12%, and fewer failed the course (Esteves, Pereira, Veiga, Vasco, & Veiga, 2017). Similarly, medium achievement and novice learners at a High School in Taiwan learning Chinese had a significant improvement on a post-test using Kahoot! compared to other groups (p < .028) (Y.-H. Wang, 2017aWang, , 2017b. Another study showed a 12% higher exam result for the students who were playing Kahoot! compared to those who were exposed to traditional teaching (p ¼ .039, effect size small) (Boboc, Orzan, Stoica, & Niculescu-Ciocan, 2018). Finally, the final exam scores of a business course with 96 students were, on average, 79.56 for students who were taught using Kahoot! compared to 56.83 for students having traditional teaching (p < .0001, effect size large) (Bawa, 2017).
Our literature review discovered two studies that compared the learning effect of using group discussions vs. using Kahoot! in class. A quasi-experiment over ten weeks in an English course in Turkey with 43 college students divided into a control group doing group discussions and an experimental group using Kahoot! (Asmalı, 2018). These groups were compared using a pre and post-test. The results showed that the control group got an average score of 46.05, and the experimental group an average score of 63.15 (p ¼ .035). A quasi-experiment in the USA with 49 undergrad students investigated the impact on the final grade of learning of lectures with group discussions vs. lectures with Kahoot! where it was found that Kahoot! made a significant impact on the exam score (p ¼ .008) (Iwamoto et al., 2017).
Some studies compare Kahoot! to the use of other tools or teaching mediums. A quasi-experiment with 77 students in an English course at university in Taiwan compared the effect of using a paper quiz vs. using Kahoot! to review learning at the end of lessons (Wichadee & Pattanapichet, 2018). The results showed that those students using Kahoot! for reviewing lessons performed significantly better (þ14.2%) than those using the paper quiz (p ¼ .000). A quasi-experiment in Turkey with 46 eight-graders tested the learning effect of using two different student response systems (Turan & Meral, 2018). One grouped used Socrative, and the other used Kahoot! over four weeks. The results of the experiment showed that the group using Kahoot! performed significantly better on the post-test (67.39 for Kahoot! and 58.70 for Socrative), which is statistically significant (p ¼ .030). In another study in an English course in Indonesia involving 77 students, a pre-and post-test was used to compare the use of textbook vs. use of Kahoot! for teaching. The results showed that Kahoot! performed significantly better than using the textbook (p ¼ .02, effect size large) (Chotimah & Rafi, 2018a). Finally, an experiment compared how the use of Kahoot! and a multiple-choice approach affected the final grade (Guardia, Del Olmo, Roa, & Berlanga, 2019). This experiment was carried out on 104 university students in Spain and showed that the final grade was significantly improved for those students playing the Kahoot! (p < .001).
Our literature review also found other articles without any statistical analysis that reports improved learning from Kahoot! such as improvement in test scores from 66% to 73% (Medina & Hurtado, 2017), improvement in test scores from 78.9% to 80.8% (Harrelson, 2017), 30% of the students improved their test scores between 40% and 55%, improvement for Kahoot! compared to traditional teaching for various groups (Głowacki, Kriukova, & Avshenyuk, 2018), and a control group that improved þ0.5% from pre-to post-test while the experimental group with Kahoot! improved þ9.5% (Iruela & Neira, 2018).
There are also a few other studies related to learning in Kahoot! that do not compare the learning effect in Kahoot! to other approaches. One example is an article investigating how the learner as a leader strategy, where students create and host their own quizzes in Kahoot! affects the learning outcome (Y.-H. Wang, 2017aWang, , 2017b. The main finding was that the learner as leader strategy contributed to enhancing discussions in groups, especially for the leading group, and benefited those who acted as leaders. Another study showed that the number of Kahoot! quizzes the students played affected their final grade, and a tendency that the more quizzes they had played the better they performed on the final exam (T� oth, L� og� o, & L� og� o, 2017). Similarly, an article investigated the relationship between the scores on Kahoot! games in a class and the final grade and found that there was a strong correlation between game performance and final grade (p ¼ .005, effect size large) (Nicolaidou, 2018). A case study from Malaysia focused on how well Kahoot! supported Nicol and Milligan's principles for good feedback practice and found that Kahoot! fulfills four out of seven principles (Omar, 2017). The identified feedback practices not fully supported in Kahoot! were 1) Ability to assist in clarifying what good performance is; 2) Ability to deliver high-quality information to students about their learning; and 3) Ability to encourage teacher and peer dialogue around learning. Finally, there is a pilot study from South Korea with 51 university students that found that Kahoot! improves vocabulary retention while fostering a positive learning environment and a meaningful learning experience (Taylor & Reynolds, 2018).
The main conclusion from the studies described above shows that Kahoot! can have a positive effect on learning compared to other tools and approaches and for various contexts and domains. All studies that include statistical significance tests and effect sizes support this conclusion. However, there are also a few studies where Kahoot! did not result in improved learning.

RQ2: Classroom dynamics
This section focuses on how Kahoot! affects classroom dynamics. Thirty-four of the articles in our literature review explicitly describe how Kahoot! affects the classroom dynamics, but only four of these contain any analysis for statistical significance. A study with 252 students at the Norwegian University of Science and Technology compared the students' perceptions from using Kahoot! the first time (once) and after using Kahoot! extensively in every lecture for five months in a software architecture course (Wang, 2015). The results showed that the only statistically significant change was for classroom dynamics (small effect size), where Kahoot! had less impact after five months compared to the first time. Another quasi-experiment at the same university with 593 students, investigated how the students' perceptions were affected by the usage of points and music/audio playing Kahoot! (Wang & Lieberoth, 2016). The results showed that the use of audio and music in Kahoot! had a significant positive impact on classroom dynamics, which resulted in an improved atmosphere for oral questions and discussions in class. The results from a quasi-experiment with 44 students in a flipped classroom English course in Taiwan showed significantly more positive responses when Kahoot! was used for interactivity with the teacher, interactivity with peers, attendance, and participation (p < .05, effect size large) (Hsiu-Ting Hung, 2017aHung, , 2017b. Especially, interactivity with peers had a low p-value (p ¼ .002, effect size large). A quasi-experiment with 77 students at Bangkok university comparing using a paper quiz with using Kahoot! to review learning, showed that Kahoot! significantly positively impacted the motivation to actively participate in class positive (þ16.5%) (Wichadee & Pattanapichet, 2018).
Our literature review found many articles that reported on improved classroom dynamics without backing it up with statistical analysis such as 100% of students reported that Kahoot! positively impacted engagement in class and 93% said Kahoot! increased their interaction and involvement in lectures (Licorish et al., 2018); 70% stated they were more actively engaged in the classroom when Kahoot! was used (Asa'd & Gunn, 2018); 95% of students agreed that Kahoot! created a positive classroom environment, and 93% agreed that Kahoot! improves the classroom atmosphere (Aktekin, Çelebi, & Aktekin, 2018); and 85.7% agreed that Kahoot! made it easier to answer questions in class (Mahon, Lyng, Crotty, & Farren, 2018). There are also studies where students have used the Likert's scale to respond to statements related to the use of Kahoot! such as "I communicate more with my friends using Kahoot!": 4.44 of 5 (Bicen & Kocakoyun, 2018); and "the use of Kahoot! helped me engage more in classroom learning": 4.40 of 5 and "I actively participated in learning activities facilitated by Kahoot!": 4.40 of 5 (Quadir, Chen, & Zhang, 2018). The literature review also found several articles that reported observed changes in classroom dynamics from the teachers perspective when using Kahoot! in the classroom (Bawa, 2017(Bawa, , 2018Cutri et al., 2016;Iwamoto et al., 2017;Susanti, 2017).
The studies related to classroom dynamics indicate that Kahoot! has a positive effect on classroom dynamics. There are only four articles that include statistical significance tests and two that report effect sizes, so more experiments are necessary for a solid conclusion. However, we did not find any studies that reported that Kahoot! negative or neutral effect on classroom dynamics. The only negative result was that Kahoot! had a less positive effect on the classroom dynamics when comparing the first time and after several months (Wang, 2015).

RQ3: Student anxiety
Some teachers are reluctant to use Kahoot! in their classroom, as they are afraid that students will get anxiety from a game that focuses on points, scoreboards, and winning. This section will present results from articles that investigate how the game-based learning platform affect student anxiety. Our literature review found fourteen studies where student anxiety is mentioned, where two tested for statistical significance. In a study comparing the use of the student response system Socrative with Kahoot! among 46 seventh graders in Turkey, it was found that there was a statistically significant difference in the anxiety scores for Kahoot! compared to Socrative (p ¼ .014) (Turan & Meral, 2018). The anxiety score for Socrative was 2.238, and for Kahoot! it was 1.871, where a lower value represents less anxiety. This result is interesting, as some teachers prefer to use Socrative to Kahoot! as the belief is that Kahoot! will produce higher anxiety among the students. Another study involving 39 ninth graders in Taiwan, where the experimental group used Kahoot! and the control group did not, the experimental group was found to have a statistically significant lower anxiety level compared to the control group towards the topic of the course -Earth Science (Lee et al., 2019).
The articles on Kahoot and student anxiety that did not include any statistical analysis include findings such as 56% of students following traditional teaching were afraid to ask questions in class compared to 37% for those who played Kahoot! in class (Budiati, 2017); students rated the statements "Does Kahoot! build student courage" to 4.46 of 5 (Bicen & Kocakoyun, 2018); and "Kahoot! does not make me stressed 2.76 of 4" (M. . Articles also include statements either from teachers or students on how Kahoot! affects student anxiety: "Students enjoyed the quiz and did not feel anxiety because it was interesting" (Susanti, 2017); "It involved people in class that are scared to speak out to tell the right answer when you ask questions" (Bawa, 2017); "Helps release stress and tension" (Muhridza, Rosli, Sirri, & Samad, 2018); "Contribute to feel safer when responding to questions and encourages participation without being judged" (Licorish et al., 2018); "Kahoot! takes the stress out of learning, adds humor to class" (Nkhoma, Nkhoma, Thomas, Tu, & Le, ); and "Kahoot! facilitated more robust participation, even by shy students. It involved people in the class who are scared to speak out to tell the right answer when you ask questions" (Bawa, 2018). However, there was also one study where the most frequently mentioned negative aspect with Kahoot! was class agitation (Moutinho & S� a, 2018).
Seventy percent of the articles focusing on student anxiety describe studies that report a reduction of student anxiety from playing Kahoot! in class, but there were only two studies with tests for statistical significance. Only one study reported that Kahoot! could produce anxiety (agitation).

RQ4: Students' perceptions
Most of the articles found in our literature review investigate how students perceive the use of Kahoot! in the classroom, and eightytwo articles reported findings from the students' point of view. Seventeen of these articles include a statistical significance test related to students' perceptions of Kahoot! One result from these articles shows that playing Kahoot! frequently over five months did not affect the students' engagement, motivation, concentration, and perceived learning negatively compared to the first time it was played, but the classroom dynamics were affected (Wang, 2015). Another study found that the use of audio and points in Kahoot! significantly affects (large effect size) the students' concentration, engagement, enjoyment, motivation, perceived learning, and classroom dynamics and that using Kahoot! without points and audio gave the worst result (Wang & Lieberoth, 2016). One study comparing using a paper quiz, a Clicker student response system and Kahoot! for reviewing knowledge, revealed significant improvement in motivation (small effect size), engagement (medium effect size), enjoyment (small effect size), and concentration (medium effect size) for the gamified approach, but not significant learning improvement . Other significant findings include the claim of getting new knowledge from playing Kahoot! compared to traditional teaching (Stoyanova, Tuparova, & Samardzhiev, 2016); significantly improved (large effect size) attendance, participation, motivation, attention and satisfaction (Hsiu-Ting Hung, 2017a, 2017b); compared to Quizizz and Google forms significantly higher concentration, engagement, enjoyment, perceived learning, motivation and satisfaction (Chaiyo & Nokham, 2017); significant improvement (large effect size) in learning culture but not significant effect on willingness to communicate (Hsiu-Ting Hung, 2017aHung, , 2017b; no significant effect was found on student motivation on using Kahoot! in learning the Chinese language (X. Liu & Wang, 2017); students using Kahoot! to review learning at the end of lessons had a significantly higher learning motivation than those who used a paper quiz (Wichadee & Pattanapichet, 2018); students using Kahoot! were significantly more engaged than students using Socrative (Turan & Meral, 2018); significantly higher perceived learning and motivation (Dolezal, Posekany, Motschnig, & Pucher, 2018); significantly higher course satisfaction (Dolezal et al., 2018); significant increase in motivation after starting to use Kahoot! (Hou, 2018); increased concentration and motivation, but not a significant increase in overall learning motivation (Lee et al., 2019); students' perception of game's impact on their learning positively correlates (medium effect size) with course grades (Nicolaidou, 2018); in a comparison of Clicker and Kahoot! -the engagement was significantly better with Kahoot! and answering carefully was significantly better with Clicker (Jones, Harden, Rassias, & Abourashchi, 2018); and Kahoot! had a more positive impact on students' engagement than Quizizzbut not significant (G€ oksün & Gürsoy, 2019). The studies above show that for the majority of studies, Kahoot! had a significant positive effect on motivation, concentration, perceived learningbut not for all studies.
Several studies do not include any statistical analysis that reports on how students perceived the use of Kahoot! in their learning. Many articles report that the majority of students perceived that Kahoot! increased their motivation and enjoyment in the classroom . Some studies report on user-friendliness, and Kahoot!'s inviting user interface and music (Bryant, Correll, & Clarke, 2018;Plump & LaRosa, 2017;Çetin, 2018). Further studies report that students found Kahoot! to be helpful to prepare for the exam (Iwamoto et al., 2017), that Kahoot! increased their interest in learning more about what they had learned and that they told others about it (Budiati, 2017), that the students motivation and confidence increased, and they felt more secure and safe in class (Tsymbal, 2018), and that students felt comfortable answering anonymously in class making the classroom-friendly and fun (Cutri et al., 2016). The main challenges reported by students included technical problems such as unreliable internet connections, hard to read questions and answers on projected screen, not being able to change answer after submission, stressful time-pressure for giving answers, not enough time to answer, afraid of losing, and hard to catch up if an incorrect answer had been given. The challenges mentioned by most articles were unreliable and slow internet connections.

RQ5: Teachers' perceptions
This section presents findings on the teachers' perception of Kahoot! from the ten articles focusing on the theme found in our review. Only one study, with 15 pre-service teachers in Biology education in Turkey, included a statistical analysis of the results related to teachers' perception of the use of Kahoot! (Yapıcı & Karakoyun, 2017). The study showed that teachers' motivation increased significantly after starting to use Kahoot! (p ¼ .001) and that Kahoot! enhanced their teaching, was entertaining, resulted in better teaching, and increased attention and concentration. Negative comments were related to persons failing the quiz and lack of technical infrastructure. In a survey, using Likert's scale among 149 secondary education teachers in Greece who were introduced to Kahoot! the most positive findings were that Kahoot! was easy to use (4.81 of 5), it was a motivating tool for learning (4.63 of 5), support teachers' instructive work in class (4.01 of 5), assess students' knowledge (4.64 of 5), and is an exciting tool for teaching (4.45 of 5).
Similarly, a study among 65 Preschool teacher students in Turkey evaluated Kahoot! very positively, including its ability to increase students' interest in the lesson, increase students' motivation, helping students' to understand the lesson better, and that it encourages learners (Bicen & Kocakoyun, 2018). Other positive perceptions about Kahoot! found in various studies include stimulating students to speak out their point of view in class and wake up students from their slumber in class (Susanti, 2017), can reduce the teacher's workload (de Sousa, 2018), can increase class participation (Parra-Santos, Molina-Jord� a, Casanova-Pastor, & Maiorano-Lauria, 2018), provides feedback and can check students' understanding, allow the lecturer to engage with a large number of students (Nkhoma et al., ), develop reading skills (Çetin, 2018), team-based kahoots help learners share knowledge and Blind kahoots encourage interest in new topics (Atherton, 2018), and well-designed kahoots lead to better student engagement motivation and learning, but poorly designed question can have to opposite effect (Smith & Brauer, 2018). The challenges or negative issues related to Kahoot! mentioned by teachers in these studies included the challenge of getting the difficulty of questions and answer time right, problems related to network connectivity, scoring based on how quickly the students answer can reduce student reflection and cause some students to guess without thinking, some students can have a problem with failing a quiz, and some teachers found it challenging to use the technology.

Discussion
This section discusses the results presented in the previous section and discusses some threats of validity.

Discussion of the results
The results of our literature review present mainly a positive view of the effect of using Kahoot! for learning. The theoretical foundation for the game concept in Kahoot! was based on Tom W. Malones theories on intrinsic motivating instruction that focuses on the three categories challenge, fantasy, and curiosity (T. W. Malone, 1980). The fantasy in Kahoot! is that the classroom is temporarily transformed into a game show, where the teacher is the game show host, and the students are contenders. This fantasy is enhanced through audio and graphics, points, scoreboards, and podium, that should contribute to a fun and positive learning experience. Several studies suggest that the game show elements, graphics, points and, audio contribute to a more positive learning environment (Abidin & Zaman, 2017;Aktekin et al., 2018;Baydas & Cicek, 2019, pp. 1-17;Lee et al., 2019;Moutinho & S� a, 2018;Susanti, 2017;Taylor & Reynolds, 2018;Turan & Meral, 2018). A crucial part of Kahoot! is the social interaction (Sweetser & Wyeth, 2005) that both take place in a digital game as well as in a classroom. Some studies in our review reported improved and increased the interaction among students (Antoniou et al., 2016;Cutri et al., 2016;Esteves et al., 2017;Hou, 2018;Mustafa, Hussein, & Zulkifle, 2018;Y.-H. Wang, 2017aY.-H. Wang, , 2017b. Challenge is related to a goal with uncertain outcomes that can be designed through variable difficulty level, multiple level goals, hidden information, and randomness. In Kahoot! the challenge is to answer a multiple-choice question correctly and as quickly as possible, and there are uncertain outcomes in terms of if the answer was correct and how many points were awarded. The correct answers are hidden until all students have given their answers. This stage of the game creates a suspension that can increase enjoyment, motivation, and concentration (Chaiyo & Nokham, 2017;Cutri et al., 2016;de Sousa, 2018 Wichadee & Pattanapichet, 2018). At the same time, student anxiety can be decreased through having the option to be anonymous both to fellow students and the teacher through the use of nicknames (Lee et al., 2019;Susanti, 2017;Turan & Meral, 2018). Several studies reported the importance for students to be anonymous, which can result in reduced stress and a safer environment to fail (Cutri et al., 2016;Jamil, Fatima, & Saeed, 2018;Jones et al., 2018;Licorish et al., 2018;Mahon et al., 2018). Curiosity is in Kahoot! stimulated through sensory curiosity through animated graphics, audio and music, and cognitive curiosity through revealing whether an answer is correct or not. Several studies mentioned positive effects from the use of graphics, audio and music in Kahoot! (Baydas & Cicek, 2019, pp. 1-17;Bicen & Kocakoyun, 2018;Bryant et al., 2018;Głowacki et al., 2018;Ismail & Ibrahim, 2018;Jamil et al., 2018;Wang & Lieberoth, 2016). The aim when creating Kahoot! was to create a learning platform that was so engaging, fun, and motivating that it would positively affect the learning outcome, classroom dynamics, and reduce student anxiety. The results of this literature review may suggest that there is a relationship between engagement, motivation, and having fun and learning outcomes and classroom dynamics.
Our review also revealed that the strong focus on competition and being a game might have some adverse effects. One challenge mentioned by four articles was stressful time-pressure for giving answers and the tendency that some students simply guessed the answer in order to try to get a high score (Bicen & Kocakoyun, 2018;Moutinho & S� a, 2018;Muhridza et al., 2018;Plump & LaRosa, 2017). Points in Kahoot! were introduced to provide a goal and a challenge in Malone's framework on intrinsic motivation (T. W. Malone, 1980). The problem here is not that Kahoot! uses points, but rather that points are awarded based on how quickly a correct answer is given. One solution could have been to just give points for correct answers, not based on timing. However, this would remove the uncertainty suggested by Malone as there is less ambiguity in how the scores are given. Another challenge mentioned by two articles was the fear of losing (Głowacki et al., 2018;Yapıcı & Karakoyun, 2017). One remedy to reduce the fear of losing recommended by three articles is to play Kahoot! as teams and not as individuals (Abidin & Zaman, 2017;Atherton, 2018;Muhridza et al., 2018).

Threats to validity
The literature review presented in this article was carried out according to the review method presented in (Dybå & Dingsøyr, 2008) to ensure maximum literature coverage, include all relevant studies and collect and synthesize the data from these studies in a meaningful way. One potential threat to validity is the coverage of the literature. In our case, the search string was simple, and should not be a source for leaving out articles. There might be some studies we did not find in our literature search, but we believe the by searching Google Scholar, Science Direct, Wiley InterScience, Web of Science, and Scopus, the coverage should be satisfactory. However, several articles were left out because they were behind a paywall where most of them were published in the proceedings for the International Conference on Technology, Education and Development (INTED).
Another possible threat to validity is the filtering of the articles found in the initial search. The filter used was to check if the articles described a study, and then articles with studies were checked for relevance (scope), rigor, and credibility. To our knowledge, all of the rejected articles failed to meet these requirements, and lacked a description of the research context, lacked a description of results, analysis of data, having conclusions not based on data, or had significant flaws in set-up or execution of the study. This process was carried out independently by the two authors of this article. A complete overview of the accepted and rejected articles with reasons for why they were rejected can be found at https://bit.ly/2ljrh5l.
One threat to validity, which is very relevant for this study, is a biased selection of articles and that only positive studies were accepted. Of the accepted articles, 97% present mainly positive results related to Kahoot! and 8% include challenges and problems. The rejected articles with a negative bias on Kahoot! included a study at a K-5 School, where they used Kahoot! and Quizizz with so many technical problems and difficulties that the results were invalid (Cadieux Bolden, Hurt, & Richardson, 2017), a presentation (not a study) of a prototype strongly inspired by Kahoot! using iBeacon to address the shortcomings of Kahoot! (Tsai, Hou, Yong, Chiou, & Yu, 2018), and a study of the effect of a flipped classroom using instructional videos, Moodle and Kahoot! where the contribution of Kahoot! was unclear (Wolf, Wilhelm-Weidner, & Nestmann, 2018). Both involved in the critical appraisal process agreed on what articles to reject, although the reasoning for rejecting the two articles was different.
Finally, the generalization of the results of this review is limited to Kahoot! and not to game-based learning or any GSRS in general, and for the context of K-12 and higher education. No studies were found on using Kahoot! for business training or other contexts.

Conclusions
This article has presented a literature review on the effect of using Kahoot! for learning, where 93 studies were included. The goal of the article was to find answers to the following five research questions: Research question one asked about the learning effect from using Kahoot! and forty-eight studies were found that covered this topic. The main conclusion is that Kahoot! can have a positive effect on learning compared to traditional learning and other learning tools and approaches and for various contexts and domains. It was found that Kahoot! had a positive effect on learning both for K-12 and higher education, as well as for language learning, technical and engineering fields, science, math, business, and nursing. However, there are also few studies where Kahoot! did not have a significant positive effect on learning performance.
Research question two focused on how Kahoot! affects classroom dynamics. The literature review included four studies that reported statistically significant results related to classroom dynamics, where one showed that the effect on classroom dynamics was reduced from first-time use to after frequent use over five months. However, the thirty-three other papers all reported on improved classroom dynamics from using Kahoot! in the classroom, including the improved teacher-student interaction and student-student interaction, more favorable to actively participate in class, improved classroom atmosphere, and easier to answer questions in class.
In research question three, the focus was on how Kahoot! affects student anxiety. Fourteen articles were found that contain results related to student anxiety, and ten articles reported that Kahoot! reduced student anxiety related to asking questions, reduce stress and tension, encourages participation without being judged, add humor to class, and enables shy students to get involved. Two studies showed a statistically significant reduction of student anxiety, and one study reported that Kahoot! could produce agitation.
Research question four asked about the students' perceptions of using Kahoot! for learning. The majority of the studies (eighty-two) in our literature review included results related to how students perceived Kahoot! The main conclusion is that students have a very positive perception of the use of Kahoot! in learning, and these results include a positive effect on motivation, engagement, concentration, perceived learning, attention, enjoyment, satisfaction, and confidence. The main challenges reported include technical challenges, hard to read questions and answers on a projected screen, time pressure, afraid of losing, and hard to catch up if an incorrect answer had been given.
The final research question five investigated how Kahoot! affects the teachers' perceptions, and the studies mainly reported that most teachers were positive about using Kahoot! for teaching. Positive findings include higher motivation of teachers, ease of use, a motivating tool, support teachers' instructive work in class, can assess students' knowledge in real-time, increases student motivation, stimulate students to speak their point of view in class, can wake up students, increase class participation, and reduce teacher's workload. The main challenges were found to be technical issues and challenges, get the questions and answers right and scoring based on timing.
The main conclusion of our literature review is that Kahoot! has a positive effect on learning, but there are challenges and room for improvement. Our literature review shows that there have been conducted several experiments on the learning effect of using Kahoot! but there is still room for empirical studies, especially on classroom dynamics, student anxiety, and perceptions of students and teachers. Further, so far, we did not find any studies on the use of Kahoot! in corporate training or similar.

Acknowledgment
Thanks to Institute for Software Research (ISR) at University of California, Irvine for hosting a research visitor and providing a stimulating research environment, Fulbright for the research scholar program for financial support, and Department of Computer Science at the Norwegian University of Science and Technology (NTNU) for moral and financial support for a research visit in California where the work on this article took place.    .

Appendix B. Creation of Kahoots
One of the most critical components of the Kahoot! platform is the creator tool that lets the user create what is named a kahoot that can contain a Quiz question, a True or false question, an Open-ended question, a Puzzle, a Poll, a Word cloud or a Slide. It is also possible to use a question bank for adding a question, which will find relevant questions based on a search on the topic. Another option is to import questions and answers from a spreadsheet. The creator allows for setting a time limit for each question (from 5 to 120 s), specify the number of points awarded for one question, add media content (picture or a YouTube-clip), and reveling the picture over time. Fig. 4 shows a screenshot from creating a Quiz question (multiple-choice). Other game-related question types are True or false questions that let players decide if the statement is true or false, Open-ended questions that ask players to type the correct answer, and Puzzle questions that ask players to place answers in the correct order. The score for these types of questions awarded for correct answers and how quickly the answers are given. None game-related question types include Poll questions that gather player opinions, Word cloud questions that collect free-form answers, and Slide that give players more context or additional explanation. The creator tool allows for adding descriptive information about the kahoot, such as a title, a description, what the intended audience is, the language, and a cover image. It is also possible to add a link to a YouTube-video that will be played in the background when the students are joining the game. The sidebar to the left in the creator allows for adding, re-arranging, duplicating, and deleting questions. Finally, the creator tool allows for choosing who should have access to the created content.

Appendix C. Play Kahoot! in the Classroom
To play Kahoot! the host (typically a teacher), will launch Kahoot! in a web-browser on a digital device connected to a large screen everyone in the room can see. When launching the game, the host has several choices that will affect the game, such as the choice of playing the game as player vs. player or team vs. player and selection of lobby music. The launch configuration also allows for enabling/disabling a name generator (students cannot pick their nicknames), randomize the order of questions, randomize the order of answers, 2-step join, display game pin throughout game, show minimized intro instructions, automatically move through questions, and require players to re-join after each kahoot. The list of game-modifiers has grown over the years based on feedback from users: The name generator was introduced to prevent students using inappropriate nicknames, and the 2-step join was introduced to avoid outsiders join and "crash" a game. In the development of Kahoot! it has always being essential to make sure that the learning experience is fun and engaging for the students, and at the same time, give the teachers control over the experience.
After launching the game, the next step is for the students to join the game. The students do not need to log into Kahoot! only enter a game PIN (unique ID for that particular game) and a nickname using a web-browser or the Kahoot! app. As soon as the students have entered their nicknames, they will appear animatedly on the lobby screen, and a counter showing the number of players will be updated. If the host (teacher) notice any inappropriate nicknames, she or he can kick-out that player. A Kahoot! lobby tune will be played during the lobby session, which puts the students in the mood for the game. The game starts with the teacher clicking on the "Start" button on the lobby screen. The game session moves through three stages, as shown in Fig. 5. First, only the question will be shown on a separate screen not shown in the figure, then the questions and answers will be shown as the screenshot to the left in Fig. 5. This screen shows a countdown clock and the number of answers given. At the same time, music adjusted to how much time the students have to answer is played, which also includes a count-down tick for every second. The music gets more intense at the end of the answering time to encourage the students to give their answers before the time is up. The teacher can also end the answering time by clicking on the "Skip"-button.
The second stage is the screen in the middle of Fig. 5, which shows the distribution of answers along with which answer(s) was/were correct (there might be multiple correct answers). The third stage is the scoreboard, presenting the top five players (screenshot to the right in Fig. 5). Occasionally, the scoreboard celebrates players who are not on top five, e.g., students answering multiple correct answers in a row, or students making a comeback. In this phase, the individual student screen will show the student's score for the last question, the total score, the rank, how far she or he is behind the player ahead, and the answer streak. A kahoot game will go through these three stages for every question until having played through all questions. It is also possible for the host to end the game early (skip questions). The game will end in a podium, and the students can also rate the quiz with one to five stars, if they learned something, if they recommend it, and if they feel happy, indifferent or sad. In the end, it is possible to replay the game using Ghost mode, where, in addition to the 'live' class, all players are joined by their ghosts (Susanti, 2017).
Kahoot! provides a downloadable mobile app for iOS or Android devices. The app enables students to create kahoots, follow their progress, and it makes it possible to play Kahoot! anywhere and at any time. The latter means that the app makes it possible to play against other players asynchronously or play against non-playable characters (NPC). In the context of school, the asynchronous and NPC play is used for homework, and it gives students an opportunity to ace games played in a class where they answered incorrectly on one or more questions. The app motivates students through avatars and achievements. Fig. 6 shows five screenshots from the mobile app, showing from left to right screens for discovering kahoots, creating kahoots, leaderboard and progress, playing kahoot, and progress for acing kahoots. In addition to the functionality mentioned above, the platform provides other features such as creating challenges, sharing kahoots, reporting, previewing kahoots, organizing kahoots in folders, favorizing kahoots, duplicating own or kahoots made by others, collaborate creation of kahoots, and team management.

Appendix D. Uses of a Game-based Student Response System
There are many ways of using a game-based student response system (GSRS) in class and out of class. It can be used at the beginning A.I. Wang and R. Tahir of a session as a starter or doing a survey, or as a recap halfway in the session to break up the lecture (Iona, 2017). Woodard and Mabry found that the most useful usages were for reviewing old and new material, obtaining baselines on prior knowledge, providing an interactive component to lecture, and for starting a discussion (Woodard & Mabry, 2018). Similarly, GSRSs can be used for formative assessment to practice skills, increase retention, and review before tests (King, 2017). To get an impactful learning experience, the teacher must carefully think through the learning goals, questions, and answers (Graham, 2015). This approach is also in alignment with creating an enjoyable gaming experience where the challenge level is essential to obtain high engagement and immersion (T. W. Malone, 1980;Sweetser & Wyeth, 2005). The main benefit of using a GSRS is to boost students' excitement, encourage curiosity and involvement for a topic, identify knowledge gaps, and use wrong answers as a teaching opportunity (Dellos, 2015). Kahoot! can help students with test anxiety to build confidence and improve their self-esteem (Johns, 2015).
The most common way of using Kahoot! is to review something that has previously been taught. However, Blind kahoots are used to introduce new topics playing through a quiz where the students do now know much about the topic from before (Castle, 2015). The approach is in a way similar to using slides for teaching, but the main difference is that a Blind kahoot stimulates the students' curiosity by first asking questions before learning more about the topic. One of the main benefits of using Blind kahoots instead of slides is that the students will interact with the topic regularly throughout the lecture. A Blind kahoot is designed similarly to a slide-set with text and illustrations, but instead of stating factsquestions are used. It is also recommended to have questions with multiple correct answers to increase the chance for the students to get a correct answer, as well as enabling the teacher to go through multiple correct answers. When using Blind kahoots in education, there should be sufficient time between the questions for the teacher to explain and go into detail. The way of using Kahoot! that gives most learning is, however, to ask the students first to create their own kahoots and then let them host them. Creating good questions demands the students to learn a subject well, as they have to come up with relevant questions, the correct answers, and the most challenging part, come up with "realistic" incorrect answers (de Sousa, 2018). This process requires students to think critically (Bryant et al., 2018).