Measuring Student Involvement When Taking Tests in E-learning Courses

The ongoing SARS-CoV-2 pandemic has significantly affected the education process of young people in primary, secondary, and tertiary education, as well as those who improve their competences through various courses and trainings. The practically incessant lockdown in education has resulted in a situation in which distance learning, which most often plays a supplementary role, now constitutes the basic form of education. This situation has forced the selection and adaptation of the available e-learning tools to the types of classes and levels of education. Despite great progress in the development of e-learning technologies, constant stimulation and maintenance of students’ interest in the content presented and monitoring their activity still pose a problem, which also concerns the education of computer science students in issues related to computer graphics. Currently, the preferred model of distance learning is based on various technologies and IT tools that enable the implementation of synchronous and asynchronous work. Prompt acquisition of data on students’ activity allows trainers to adjust the form and content of the presented material to their expectations. It also makes it possible to identify issues that pose great difficulties to students. This paper discusses modern solutions and trends in designing e-learning courses based on the new interactive e-learning platform Quizer. The aim of the research is to determine the course ef - fectiveness using the user activity-monitoring module designed and implemented for the Quizer e-learning platform. The correctness of the answers correlates with

the number of attempts to answer it (−0.65).The time devoted to a given question was usually high for questions with low correctness of answers.K e y w o r d s: e-learning, e-learning platform, e-learning course, activity monitoring The current challenging situation related to the spread of SARS-CoV-2 virus worldwide has significantly limited interpersonal contacts.This also applies to the education system, where education at schools and universities has been replaced by distance learning, often becoming its basic form (Barszcz, Montusiewicz, Nowicki, & Kayumow, 2021).The recurring lockdown makes it far more difficult for students to return to school and university.Most Polish universities (62.4%) declare that they have implemented e-learning in recent years, but only 49% (of all universities) have confirmed the targeted selection to satisfy specific needs (Maleńczyk & Gładysz, 2019), which may hamper the adaptation of their resources to the transition to remote learning.
The concept of distance learning is the broadest concept because it includes correspondence learning, television and radio forms, and e-learning (Szulc, 2020).At schools and universities, e-learning is mainly used for distance learning.The use of information technology has made it possible to replace real contact with a virtual one.In literature, e-learning is understood as a process that enables the relationship between the learner and the teacher through achievable technical means (usually digital ones) (Duda, Korga, & Gnapowski, 2014).In practice, elearning involves teaching via the Internet with the use of personal computers or portable wireless devices -smartphones, tablets, or laptops.The use of dedicated e-learning applications and platforms enabling communication and teacher-student data exchange has become a necessary requirement in distance learning.Over the last years, their rapid development has been observed.E-learning applications and platforms have become very extensive, offering a number of various functionalities in line with the development of technology and teaching methods (Sierra & Sarasa-Cabezuelo, 2014).
The positive aspects of using e-learning in the process of educating pupils and students are discussed in numerous scientific papers.Improvement of 'hard skills', such as the use of new technologies and IT tools, as well as 'soft skills', such as communication and presentation skills, is observed (Csikosova, Senova, & Culkova, 2012).In the literature, e-learning platforms are mainly referred to as LMS (Learning Management System) or LCMS (Learning Content Management System) platforms (Caputi & Garrido, 2015).Currently, SCORM and xAPI are the most commonly used formats for teaching content and quizzes available on e-learning platforms.
Nonetheless, new technical and functional solutions are still being sought in e-learning.Even commercial solutions frequently have the option of installing dedicated extensions that add new functionalities.One such example is the Moo-dleRec plug-in for content search.This plug-in can sort a set of supported standard repositories of educational objects and suggest an ordered list of these.For this purpose, it uses queries based on defined keywords (De Medio, Limongelli, Sciarrone, & Temperin, 2020).
Intelligent e-learning systems are an innovative approach.They enable trainers to provide education in an adaptive way, which makes it possible to take into account learners' specific needs when transmitting content.The ability to independently make decisions concerning what teaching content is to be provided to the learner in response to the identified needs is implemented with the use of the socalled teaching strategies and the IT module implemented in the LMS system, the so-called ITS agent.The key function of the ITS agent is to confront the teacher's expectations with the learner's actual progress and to make decisions as to the content provided (Marciniak, 2016).The platform used at Kajaani University of Applied Sciences is a good example of a personalised e-learning platform which uses adaptive mechanisms for customising courses for students (Deepak, 2017).
Currently, there is a trend to exploit the popularity of computer games and 3D visualization in e-learning.Children and teenagers devote a large part of their spare time to this kind of entertainment.Gamification is an approach applied in the educational process, which is now gaining in popularity.The introduction of gameplay elements into the learning process boosts the user's interest, creativity, and motivation as regards the content presented.Elements of strategy and gameplay are combined with educational content in educational games.During the game, players often have to show prompt reactions and knowledge connected with the questions asked (Montusiewicz et al., 2017).Their activity and the correctness of responses are measured and recorded for further analysis of the final effects.
Simulation games are another type of games used in teaching.They are mainly applied in higher education, in particular in the process of educating students of engineering, medical and military majors, as well as pilots (Haugea & Riedel, 2012).Modern solutions such as virtual and augmented reality, as well as artificial intelligence are increasingly used in simulation games (Marciniak, 2016).They allow for high realism of the presented content, thus increasing its understanding (Bauman, 2016).The implementation of gamification in e-learning requires a multidisciplinary approach, combining the ability to design games and knowledge of information technology with psychology and pedagogy (Urha, Vukovic, Jereb, & Pintar, 2015).
The positive effects of using a programming course with elements of gamification via an e-learning platform are presented in Pankiewicz's paper.Students who were provided with the elements of gamification achieved a better final result (than the control group), assessed the difficulty of the material as easier, and more often assessed the obtained grades as fair (Pankiewicz, 2016).Presenting content with the use of 3D computer visualisations is particularly useful in teaching technical subjects related to the design of machine parts (Dziedzic, Włodarczyk, & Paśnikowska-Łukaszuk, 2014).
To sum up, it may be stated that e-learning software continues to be dynamically developed, which will certainly be further accelerated by the global SARS-CoV-2 pandemic.

Methodology of Research General Background. Modern Technologies and Solutions Supporting Distance Learning
Currently, in the time of the SARS-CoV-2 pandemic, the most commonly used model at universities is the distance learning model which uses various technologies and IT tools that allow for synchronous and asynchronous work with students.The most commonly used commercial solutions include Moodle e-learning platform, Microsoft Teams application, and Zoom conference software.
The Moodle learning platform is one of the most popular tools used to implement distance learning.It is an open-source learning management system (LMS) written in PHP and distributed under the GNU (General Public License) license, which can be installed on university-owned, purchased or leased server infrastructure.Functionalities of the platform can be extended with the use of plug-ins, created by the developers' community around the world.The Moodle platform is most frequently used as an information board that makes it possible to place any information, e.g., information regarding the courses taught, a list of recommended literature, and as a repository of files, e.g., for posting materials for classes, homework, or reports.In addition, the application enables the use of functionalities such as discussion forums, quizzes, surveys, or checking attendance.It is very important to be able to import the created courses in SCORM (Sharable Content Object Reference Model) format, which is a modern standard that defines the method of preparing an e-learning course in the IT layer (specification of files, extensions, file headers, blocks in which data should be placed, and embedded objects).Importantly, the SCORM package is independent of the LMS system used.Such packages are usually zipped files which contain the necessary content of the course.
Microsoft Teams is a virtual, cloud-based environment with tools for multilevel and multi-threaded collaboration between team members.It is part of the Microsoft 365 platform distributed under a closed software license.It is impossible to copy, modify, and distribute the source code.In the process of education, it is most often used in the following areas: planning meetings, basic voice and text communication, videoconferencing with students, sharing a remote desktop during classes, and recording student attendance.Combined with other platform tools such as Forms, Whiteboard, OneDrive, or Outlook, it is an integrated environment which enables synchronous and asynchronous distance learning.
Zoom is another very popular solution.It is a software mainly designed for planning and conducting videoconferences with students, which operates in a browser or in an installed application.At universities, this tool is especially useful for conducting lectures and seminars.Other, less popular solutions used in Poland include Skype, Discord, Google Docs, Google Hangouts, and Slack.Most distance learning software works on both computers and mobile devices, regardless of the operating system installed.
The Quizer e-learning platform is an interesting, alternative, and modern solution, designed to create interactive multimedia courses.The platform has been developed by Euro-Forum as a result of research and development work carried out as part of the project: "Creating an innovative interactive Quizer platform as a result of R&D works".Euro-Forum draws upon it to create courses and makes it available for commercial sale for schools, universities, and institution.
It is characterised by the functionality of a multimedia course creator that enables the introduction of elements of gamification and a module that monitors user activity during quiz solving.The platform is a new solution targeted at companies and institutions, at schools and universities implementing remote learning, as well as companies and institutions that would like to improve the qualifications of their employees through training and online courses.It enables the preparation of training materials, presentations, as well as final tests with extensive statistics of final results.
Designing e-learning platforms that can operate in the cloud is becoming a standard nowadays.This solution allows for the elimination of the need to have one's own server infrastructure.The cloud infrastructure used in the case of the Quizer platform is Oktawave.It ensures its uniform operation on various devices such as PCs, notebooks, tablets, and smartphones.Scalability is a key feature for cloud infrastructure.Cloud computing makes it possible to allocate the right amount of resources depending on the situation during the platform operation.Such a solution for the e-learning platform increases its smooth operation, reliability, data security, and, concurrent, reduces the costs of infrastructure maintenance.IJREL.2021.7.1.09p. 5/17 At the same time, it is vital to use an operating system that enables stable operation of the application.In the case of the Quizer platform, the virtual application server runs on the Linux-Debian operating system.Bootstrap, PHP Laravel, and MySQL have been used to build the application itself.Bootstrap is an open source CSS (Cascading Style Sheets) library.It is one of the solutions that allows for the construction of responsive websites.A responsive website (Responsive Web Design) is a type of page that adjusts the layout of elements and its width to the screen resolution of the device on which it is displayed.
A framework is another essential tool for designing an e-learning platform, the example of which is the one used to build the Laravel platform, written in PHP and based on the Model-View-Controller architectural pattern, thanks to which developers have at their disposal a number of additional built-in functions and may develop applications at various levels of advancement.
The MySQL relational database management system was used to manage the database.Such a set of the most important modern IT tools allows for the design and implementation of e-learning platforms.A general diagram of the Quizer elearning platform structure is presented in Fig. 1.Users can set up their individual accounts and gain access to the resources available on the platform.The introduction of mechanisms supporting user engagement and motivation for e-learning can be achieved by introducing gamification.The interactive exercise wizard is one of the elements of the Quizer platform and makes it possible to create quizzes.On the Quizer platform, two solutions have been used: quizzes can be created in single-player (a single player) and multiplayer (many players) formats.This allows for the creation of educational games, an alternative to other types of games which children, in particular the youngest, most often do not benefit from.The games may be played against the computer or other users.
The single-player wizard makes it possible to create multimedia educational courses and presentations using interactions, including: single choice, multiple choice, drag-and-drop, type, crossword, puzzle, connect with lines, and more.The wizard also gives the possibility to design conditional interactions based on previous user actions, using built-in or user-defined variables.By using variables, one may also monitor and examine user activity in a given course.The quizzes prepared in the wizard can be exported to SCORM or HTML formats, which enables cross-platform exchange.In addition, it is possible to publish the quiz directly to the Presenter, thanks to which it is automatically placed on the presentation platform, which constitutes a separate component of the Quizer platform (Dziedzic, Gudków &, Wiśniewski, 2020).
User activity monitoring is an optional functionality of the platform designed to observe behaviour and to measure the level of students' involvement.One may and in Google documents (Form) focus mainly on analysing the achieved results without analysing user activity when taking tests.The Quizer platform offers the functionality of measuring user activity in its basic form.The functionalities of Moodle can be extended with plug-ins.The aim of such solutions is to ultimately increase the effectiveness of the designed courses.Access to the user activity monitoring panel is provided by a web browser and access to the Internet.The tools used to create the application include jQuery programming library, Bootstrap, PHP Laravel, and MySQL.The general outline of the structure of the application for user activity monitoring is presented in Fig. 2. Triggers are procedures performed automatically as reactions to events in a database table.This enables the measurement of user activity and the creation of statistics on the answers given.All data is collected automatically.

Sample of Research, Procedures, and Data Analysis
The basic research problem is to determine the involvement of computer science students in completing an e-course in computer graphics, which is implemented in their education process.Computer science students usually prefer to use new IT solutions in education due to the nature of their future work.However, it very frequently occurs that when filling in various questionnaires or e-tests, students do it carelessly and quickly, which distorts the obtained results.The research hypothesis is as follows "Measurement of student activity while completing an e-course should facilitate the determination of their involvement in this process".
The following research questions have been formulated: • How to measure student involvement during the e-course completion?
• Is it possible to indicate a correlation between the presented content and the time to respond and the use of elements facilitating the choice of the correct response, such as instructions and hints?• Does the way the content is presented and the colours are used affect the process of completing the e-course?
The aims of the research consist in: • characteristics of the new Quizer platform module for measuring user activity, • measurement of user activity while taking tests, • indication of relationships and correlations between recorded data, • determination of the course effectiveness using the user activity monitoring module designed and implemented for the Quizer e-learning platform.
The research was conducted on computer science students during the course of computer graphics.The subject of "Fundamentals of Computer Graphics" is introduced in the second year of studies.The first and the second group amounted IJREL.2021.7.1.09p. 8/17 Measuring Student Involvement When Taking Tests in E-learning Courses to 12 people each, while the third one comprised 13 people.The study groups consisted of both men (31) and women (6).
Before starting the test, each person was assigned a unique user number, which was used to set up an individual anonymous account on the Quizer platform.The same test was performed in all groups with assumptions as in Table 1.The closed test was the type of test used.

Table 1
The type of questions and their arrangement on the screen  It was assumed that, for adults, the time needed to complete one test should not exceed 5 minutes (300 s).Upon test completion, users answered survey questions regarding the arrangement of questions and the colour scheme.After entering IJREL.2021.7.1.09p. 9/17 the test performed by users, it was possible to collectively analyse the time and results for the entire study group.The presentation of results in the form of graphs facilitated and accelerated their interpretation.
The application also allows a detailed analysis of activity for each user.Subsequently, the results obtained were exported to a spreadsheet for further processing.Pearson linear correlation coefficient was calculated for the data.

Results and discussion
While taking the test, each user's efficiency of work was measured with the use of the user activity monitoring tool and the survey method.Among other things, the test execution time, viewing the instructions and hints, the time of displaying the questions on the screen, the number of attempts to answer, and the correctness of the answers were recorded for all target groups.
The number of attempts provides information on whether the user changed his/her decision while selecting the correct answer.Measurement started on the title page where the test was initiated.Instructions on how to do the test have been displayed on the title page.Upon test completion, the users answered survey questions regarding the question's arrangement on the page, the graphic design used, and the validity of instructions and hints.
The recorded average time of performing the test by all participants was 157.6 seconds.The maximum time of taking the test that was registered was 239 seconds.The minimum time was 89 seconds.No person exceeded the assumed maximum time for performing the test.
The list of basic parameters recorded during the test execution for users from the first group is presented in Table 2. Data for the remaining groups were recorded in a similar way.While analysing the data in the table, one may observe that, in the first group, none of the students used the hints, relying solely on their own knowledge.This was probably due to the fact that students did not receive grades for completing the test.Four people clicked on the instruction regarding how to complete the test.The time spent on the title page was generally the highest for those who used the instruction, which suggests that those persons have, in fact, read it.Similar relationships were observed for other groups.Five people used the instruction in the second group and four in the third group.The instruction display time was a maximum of 11 seconds.
The correctness of the answers given by users is shown as an average for each group for individual questions in Figure 4.For groups 1 and 2, question 5 caused the greatest problems, where the correctness of the answers was only 25%.For group 3, the most difficult question was question 4, where the correctness of the answers was 38.8%.When analysing all answers, the questions that caused problems for the students were questions 3, 4, 5, 6, 7, and 9.The least problematic questions were questions 1, 2, 8, and 10, where the correctness of the answers was over 75%.No. of attempts -question 2 6 10 14 12 5 12 8 7 15 10 11 5 No. of attempts -question 3 No. of attempts -question 4 No. of attempts -question 5 No. of attempts -question 6 No. of attempts -question 10 The average time devoted to solving particular questions and the average number of attempts for all users are presented in Fig. 5.The measured times for questions 4, 5, and 6, which turned out to be the most difficult ones, correlate (−0.55) with the correctness of the answers given by the students.IJREL.2021.7.1.09p. 11/17 At the same time, the increased number of attempts at answering is visible for these questions.This proves that students analysed these questions longer, often changing the final answer (on average more than 2 times).For the remaining questions, the number of attempts is at a similar level.The arrangement of questions did not significantly affect the answers provided; nevertheless, in the survey responses, users indicated the vertical arrangement of questions as the preferred one.When assessing the graphic design, users definitely pointed to the questions where a subdued (grey-blue) graphic design was used compared to an aggressive (red-yellow) design.

Conclusions
On the basis of the conducted research, it may be concluded that the 'User activity monitoring' application implemented on the Quizer platform allows for examining the parameters of users' activity and the results achieved by them.It facilitates the analysis and interpretation of data as well as determination of the occurring correlations.The tool makes it possible to record for each user (or groups of users) such parameters as the number of clicks on hints and instructions, correctness of given hints, time spent on the title page, time devoted to individual questions, and number of attempts for each question.
The application indicates which questions were easy for users and which were difficult for them.Such information makes it possible to analyse these questions, and thus find the causes of user's problems.This allows for the correction of the provided content during the teaching process itself, which can significantly affect the teaching process, contributing to an increase in its effectiveness.The tool enables the compilation of data obtained both for individual users and for individual groups; it makes it possible to observe which groups fared best and which were the weakest in particular questions and where the reason for such a state of affairs may lie.Based on the research, the following conclusions have been drawn: 1.The correctness of the answers correlates with the number of attempts to answer it (-0.65).The time devoted to a given question was usually high for questions with low correctness of answers.2. When analysing the answers from the survey questions, the vertical arrangement of answers seems to be the clearest/most legible.The courses designed should use subdued graphic design.In the users' opinion, graphic design is important for focusing on the task.Aggressive screen graphics distracts the user and extends the test execution time.Additionally, users expressed positive opinions on the inclusion of hints and instructions in the course.3. The user activity monitoring module allows for a wide configuration of the measured parameters, which may contribute to the correction of content included in the course and its distribution, and thus to an increase of the test's effectiveness.
S o u r c e s: Author's own work IJREL.2021.7.1.09p. 6/17 Measuring Student Involvement When Taking Tests in E-learning Courses

Figure 2 .
Figure 2. Outline of the structure of the application for user activity monitoring.S o u r c e s: Author's own work

Figure 3
Figure 3 presents an example of a test question used during the research.

Figure 3 .
Figure 3.A sample question used during the research.S o u r c e s: Author's own work

Figure 4 .
Figure 4. Correctness of the answers given for all groups.
S o u r c e s: Author's own work

Figure 5 .
Figure 5.The average time devoted to solving particular questions and the average of the number of attempts.S o u r c e s: Author's own work

Table 2
Recorded test parameters for users in Group 1