The Use of the Kahoot! Learning Platform as a Type of Formative Assessment in the Context of Pre-University Education during the COVID-19 Pandemic Period

: The present study aims to display how using a personal assessment environment based on the interactive Kahoot! platform actively supports the teaching–learning process. The goal is to improve the instructive–educational process by applying a learning platform based on play and digital technology that favors a qualitative educational endeavor. The use of the Kahoot! platform as form of assessment had a signiﬁcant and direct positive effect on the educational process during the COVID-19 pandemic. assessment is a motivating tool for teaching activities in geography classes?”; and 87.3% agreement for the question “To what extent do you think you want to use the interactive game Kahoot! in geography lessons as a feedback/summative assessment sequence in the next school year?”


Introduction
With the onset of the COVID-19 pandemic, many schools have had to approach an enforced online or hybrid teaching-learning-assessment system. Many teachers have been put in the situation of having to associate digital competences to the teaching subject in order to ensure that the educational process is being conducted in a good and effective way.
In this regard, there has recently been a large amount of research conducted on the learning, teaching, and assessment approaches that have been adopted since the outbreak of the COVID-19 pandemic and the impact of those approaches on students [1].
Several articles have demonstrated that digital learning has a beneficial influence on the intellectual development of students and teachers [2,3]. For example, the technical skills of teaching staff have increased as a result of digital learning [4]; the average duration of courses is lower [5]; the education system is better structured [6]; there has been an increase in e-learning techniques [7]; and more opportunities to give feedback to students have arisen [8].
Other studies have highlighted the inability of some schools to use online learning systems [9,10], while other research has demonstrated that some students have no competence or self-control when trained face to face [11]. A recent study also demonstrated that internet connectivity is very poor in China, with a decision being made to allow international students to remain in their student hostels in order to access their university's internet services [12].
An article produced in the context of COVID-19 used the flipped classroom (FC) teaching model, which revolutionized the field of education, did not show sufficient evidence of its advantages and disadvantages in the university setting [13].
The COVID-19 pandemic has led to changes in the development of educational processes [14]. The health rules that have been adopted at different times of the pandemic The groups of students used in the research are intellectually homogeneous and have a predominantly visual learning style. A Google Forms questionnaire was conducted for students regarding their perceptions of the interactive game Kahoot! for the feedback/summative assessment sequence in geography lessons.
The study was thus divided into three stages of research: • Step 1: A study was conducted on the standardized Kahoot! interactive exercise feedback assessments for all classes in the online teaching-learning-assessment model over five weeks of school.

•
Step 2: A study of the standardized feedback and final review assessments was conducted on the Kahoot! learning platform for all classes in a traditional teachinglearning-assessment model during six weeks of school. • Step 3: Conduct, complete, and analyze the Google Forms questionnaire for student perception of the interactive game Kahoot! for the feedback/summative assessment sequence in geography lessons.
In the 1st and 2nd phases of research, all of the students in the class had provide written answers in the Kahoot! game in the last 5-10 min of the class as a feedback sequence to 9 standardized questions according to the 5 proposed operational objectives and in accordance with the 3-5 specific competencies in the school curriculum (as an assessment scale: 9 marks in total for the questions and a mark was awarded ex officio or to tests in the online model, with a feedback question on the lesson taught by the teacher). The items used on the Kahoot! platform was of a dual choice objective type (true/false) and multiple choice/quiz (where only four answers were present and where only one choice was correct).
In stage 3, student perceptions on the use of Kahoot! As an assessment tool were collected by completing a Google Forms questionnaire and was conducted mostly on a Likert scale, with the aim of building the self-correction and self-assessment capacities of the students and of determining the likelihood of the teacher using this tool as a form of assessment again in the future. This created a database of the assessment results for each class.
The methodology used to test the hypothesis of the effect scale that was studied was the quantitative statistical calculating method to determine the expected progress that the students should have made during the research period for each learning model.

Results
In the first phase of research, standardized assessments were applied on the Kahoot! learning platform for five class hours where geography was being studied (in the 5th, 6th, 9th and 10th grades) in order to observe the level of knowledge acquired in the online teaching-learning-assessment model. The results obtained by the students in each class were synthesized and are illustrated in Figure 1  In the second stage of research, standardized assessments with the Kahoot! game were also applied for six class hours where geography was being studied (in the 5th, 6th, 9th, and 10th grades) in order to observe the level of knowledge acquired in the traditional and online teaching-learning-assessment models compared to the online only model. The results obtained by the students in each class were synthesized and are reflected in Figure 5 (5th grade), Figure 6  terms of the values of the impact size per class for the students′ school results based on a standardized assessment background, the comparative values between the classes were as follows: in the fifth grade, the averages per class on the online model ( Figure 1) were 7.64 (5th grade A) and 6.29 (5th grade B) compared to the traditional model ( Figure 5), which had averages of 6.96 (5th grade A) and 7.14 (5th grade B). It must be noted that in the Romanian education system, multiple classes of the same grade are assigned lettersthus 5th grade A and 5th grade B are both the same grade, but the students learn separately. Figure 1. Evolution of student grades in the assessment sequence of the Kahoot! interactive game; type-traditional model of teaching-learning-assessment-5th grade parallel classrooms A and B.           In the 9th grade, the averages per class in the online model ( Figure 5) were 6.88 (9th grade A), 6.32 (9th grade C), 8          The traditional model has been observed to be better than the online model in the context of grades. The positive influence regarding the traditional model is the fact that all students are carefully supervised by the teacher, all of the students take notes under direct observation and guidance, and all of the students carry out the proposed learning activities, thus improving the students′ results and highlighting their progress. This results in a higher percentage of active students compared to passive students when compared to the online model.
In the online teaching-learning system, a negative factor is represented by the poor direct supervision of the students by the teachers, the students being able to have completely different concerns during the teaching-learning management sequence, resulting in random and wrong answers during the assessment period, thus generating the lower results.
The Google Forms, active method, was combined with PowerPoint presentation, a passive method, to create a method that could be used to verify the knowledge that the students retained from the presentation, and this combined method was more effective than in cases where only the passive PowerPoint method was used. Additionally, during the face-to-face period, the summative test, where all of the content studied throughout the year is applied, the students obtained slightly lower grades . The reason for this is that students cheat less in the traditional model; thus, the grades that they obtain are closer to their current level of knowledge.
Other causes of lower results using the Kahoot method! can be caused by the poor internet connection that some students have, the degree of difficulty of the contents of the lessons being studied, and the hardware performance of the devices used.
These results highlight the fact that the variable influencing the results is the training approach, which must be rethought so that all of the students are involved in the educa- During the research with the two comparative teaching and learning models, in terms of the values of the impact size per class for the students school results based on a standardized assessment background, the comparative values between the classes were as follows: in the fifth grade, the averages per class on the online model ( Figure 1) were 7.64 (5th grade A) and 6.29 (5th grade B) compared to the traditional model ( Figure 5), which had averages of 6.96 (5th grade A) and 7.14 (5th grade B). It must be noted that in the Romanian education system, multiple classes of the same grade are assigned letters-thus 5th grade A and 5th grade B are both the same grade, but the students learn separately.
Additionally, in the 6th grade, the averages per class on the online model ( Figure 2) were 6.79 (6th grade -A) and 6.53 (6th grade B) compared to the traditional model ( Figure 6), which had averages of 7.88 (6th grade -A) and 6.86 (6th grade B). There is a significant increase in the grades of the students learning in the traditional model compared to the online one, with a maximum value of 1. 09 points in 6th grade A (13.9%), 0. 33 points in 6th grade B (4.9%), and 0.85 points in 5 grade B (12%).
In the 9th grade, the averages per class in the online model ( Figure 5 In high school classes, a spectacular increase in the results can be observed in the traditional model compared to in the online model in the 9th and 10th grades, with a maximum value of 1.37 points in 9th grade C (17.9%) and of 1.32 points in 10th grade D (16.1%).
The traditional model has been observed to be better than the online model in the context of grades. The positive influence regarding the traditional model is the fact that all students are carefully supervised by the teacher, all of the students take notes under direct observation and guidance, and all of the students carry out the proposed learning activities, thus improving the students' results and highlighting their progress. This results in a higher percentage of active students compared to passive students when compared to the online model.
In the online teaching-learning system, a negative factor is represented by the poor direct supervision of the students by the teachers, the students being able to have completely different concerns during the teaching-learning management sequence, resulting in random and wrong answers during the assessment period, thus generating the lower results.
The Google Forms, active method, was combined with PowerPoint presentation, a passive method, to create a method that could be used to verify the knowledge that the students retained from the presentation, and this combined method was more effective than in cases where only the passive PowerPoint method was used. Additionally, during the face-to-face period, the summative test, where all of the content studied throughout the year is applied, the students obtained slightly lower grades. The reason for this is that students cheat less in the traditional model; thus, the grades that they obtain are closer to their current level of knowledge.
Other causes of lower results using the Kahoot method! can be caused by the poor internet connection that some students have, the degree of difficulty of the contents of the lessons being studied, and the hardware performance of the devices used.
These results highlight the fact that the variable influencing the results is the training approach, which must be rethought so that all of the students are involved in the educational process since the students who were inattentive during the teaching process or who did not formulate answers to the learning activities during the class but who participated in the Kahoot! game assessment test achieved very poor results.
The results obtained by each class in the third stage of the Google Forms questionnaire on the student perceptions of the interactive game of Kahoot! for the feedbackback/summative assessment sequence in geography lessons are shown in Figure 9.
Thus, the minimum and maximum average values are 100% for the total disagreement scale and for student disagreement with the three questions in Figure 9; 84.9% total agreement for the question "To what extent do you consider that the interactive Kahoot! game for feed-back/summative assessment has a positive effect on learning geography?; 77% agreement for the question "To what extent do you consider that the Kahoot! interactive game of feed-back/summative assessment is a motivating tool for teaching activities in geography classes?"; and 87.3% agreement for the question "To what extent do you think you want to use the interactive game Kahoot! in geography lessons as a feedback/summative assessment sequence in the next school year?" The results obtained by each class in the third stage of the Google Forms questionnaire on the student perceptions of the interactive game of Kahoot! for the feedbackback/summative assessment sequence in geography lessons are shown in Figure 9.
Thus, the minimum and maximum average values are 100% for the total disagreement scale and for student disagreement with the three questions in Figure 9; 84.9% total agreement for the question "To what extent do you consider that the interactive Kahoot! game for feed-back/summative assessment has a positive effect on learning geography?; 77% agreement for the question "To what extent do you consider that the Kahoot! interactive game of feed-back/summative assessment is a motivating tool for teaching activities in geography classes?"; and 87.3% agreement for the question "To what extent do you think you want to use the interactive game Kahoot! in geography lessons as a feedback/summative assessment sequence in the next school year?" Regarding the question "Have you been more attentive and achieved better results during the Kahoot! interactive feed-back/summative assessment game in the online model or in the traditional teaching-learning-assessment model?", there was a minimum value of 0.9% in the response variant in both of the two teaching-learning-assessment models, with a maximum value of 59.8% attention and better grades being achieved for both teaching-learning-assessment models.
In terms of whether students desired to use the interactive game Kahoot! in future geography lessons and why, multiple reasons stood out: the Kahoot! platform is a motivating tool for study; develops competitiveness between students in the class and other classes; increases grades and attention; provides more response options; is an interesting and fun, way to learn and consolidate the information of the lesson in class; provides feedback to the teacher about who understood the lesson and who did not; and it is easier to learn through play. The students also stated that Kahoot! helped the students pay attention; the scores were a motivational factor; the game is simple to use/accessible; it has a reduced assessment time; it is a method of real-time self-assessment; it develops thinking skills; it is relaxing; it helps to conduct an interactive lesson; it is a practical, interactive, and innovative revision method and effective learning method; it is creative; it offers distributive attention and answers for all students; and it is easier than conducting a test on paper. Additionally, the students that the Kahoot! game offers a friendly type of learning; has a three-person podium that is motivating because of the social desire to be at the top; provides a ranking for all students; is educational; stimulates students to complete a productive activity; is interesting; provides helpful images; has a positive effect on the performance of students; does not promote cheating; and improves digital skills. The objectivity of the assessment of the results and the development of self-confidence induces a state of well-being. Regarding the question "Have you been more attentive and achieved better results during the Kahoot! interactive feed-back/summative assessment game in the online model or in the traditional teaching-learning-assessment model?", there was a minimum value of 0.9% in the response variant in both of the two teaching-learning-assessment models, with a maximum value of 59.8% attention and better grades being achieved for both teaching-learning-assessment models.
In terms of whether students desired to use the interactive game Kahoot! in future geography lessons and why, multiple reasons stood out: the Kahoot! platform is a motivating tool for study; develops competitiveness between students in the class and other classes; increases grades and attention; provides more response options; is an interesting and fun, way to learn and consolidate the information of the lesson in class; provides feedback to the teacher about who understood the lesson and who did not; and it is easier to learn through play. The students also stated that Kahoot! helped the students pay attention; the scores were a motivational factor; the game is simple to use/accessible; it has a reduced assessment time; it is a method of real-time self-assessment; it develops thinking skills; it is relaxing; it helps to conduct an interactive lesson; it is a practical, interactive, and innovative revision method and effective learning method; it is creative; it offers distributive attention and answers for all students; and it is easier than conducting a test on paper. Additionally, the students that the Kahoot! game offers a friendly type of learning; has a three-person podium that is motivating because of the social desire to be at the top; provides a ranking for all students; is educational; stimulates students to complete a productive activity; is interesting; provides helpful images; has a positive effect on the performance of students; does not promote cheating; and improves digital skills. The objectivity of the assessment of the results and the development of self-confidence induces a state of well-being.

Discussion
The main purpose of the research was to check to what extent the interactive game Kahoot! helps students to progress and be motivated and interested in the study of geography. It has also been shown that in the traditional model, the supervision and involvement of all students in the teaching-learning process leads to higher performance, and in the online model, the Google Forms tool method combined with a PowerPoint presentation led to the students paying attention and being motivation to learn and develop the competences in the curriculum. Formative assessment is what supports and promotes learning [38,39].
The research results showed that the method of assessment based on the use of the Kahoot! platform improves the quality of the teaching-learning-assessment process and encourages all students to participate and take on an active role in class. Therefore, the choice of the teacher regarding the teaching-learning-assessment method has a direct impact on the knowledge retained by the students. [40,41].
This study also highlighted that the standardized assessment of student outcomes using the interactive exercise Kahoot! has positive effects on learning, which is in agreement with, other studies [34] as well as on student perceptions [35,36].
This study can contribute to the completion of practical approaches regarding effective methods of assessment as well as practice approaches for teaching and learning in the pre-university environment.
The current study has limitations that need to be considered, but that could indicate potential future lines of research. One of the main limitations is the size and origin of the sample (392 students of a single Romanian high school); therefore, generalizations based on these results should be treated with caution.

Conclusions
This study aimed to prepare a list of interventions to improve types of formative assessment with the Kahoot! interactive exercise followed by the analysis of a questionnaire determining student perceptions regarding the reuse of the Kahoot! platform as the form of assessment and feedback in the final revision sequence in the next school year.
The Kahoot! game learning platform used in the assessment sequence induces neither a separation from the operational objectives of the lesson and the specific learning competencies defined in the curriculum and does not require or imply special digital skills.
The efficiency of the formative/summative assessment modality with the Kahoot! interactive game requires patience, constant and increased attention, a permanent and qualitative internet connection as well as two electronic devices (preferably for those who only benefit from the phone).
It is highlighted that the motivation, ease, and usefulness perceived by students regarding the use of the interactive game Kahoot! have a positive and significant influence on the training of the skills indicated in the school curriculum, creating an environment that is conducive to study. This method can be used in all three teaching-learning models (traditional, hybrid, and online), thus ensuring that the method remains consistent in any scenario, reducing student stress during assessments.
Recent studies on the impact of COVID-19 on educational institutions are relatively few; however, more research has proposed ways in which the transition from face-to-face teaching to online learning could be achieved, for example, digital literacy training, the use of online classrooms, encouraging students to use peer learning, and teachers focusing on improving the virtual or physical engagement of their students [42][43][44][45].
The results of this study will be of interest to the specialized field literature, providing an overview of the positive results in terms of the learning and perceptions of students on the value of the Kahoot! platform as a method of evaluation in pre-university education.
The fundamental changes that have occurred in Romanian education system requires concrete actions in order to increase the adaptive capacity of the educational system. One of the directions of action to overcome this challenge can be represented by the students' knowledge of new methodologies for the analysis of geography. Knowledge of the basic elements of the Geographic Information System (GIS) and fractal analysis can be easy means of transmitting new trends in modeling the territorial reality [46][47][48][49][50][51][52][53].
Possible future research directions could include the application of the proposed learning platform in other cultural, geographical, and economic areas and in the context of systemic crises.