Students Perception on the Use of Computer Based Test

Teaching nowadays might use technology in order to disseminate science and knowledge. As part of teaching, the way evaluating study progress and result has also benefited from this IT rapid progress. The computer-based test (CBT) has been introduced to replace the more conventional Paper and Pencil Test (PPT). CBT are considered more advantageous than PPT. It is considered as more efficient, transparent, and has the ability of minimising fraud in cognitive evaluation. Current studies have indicated the debate of CBT vs PPT usage. Most of the current research compares the two methods without exploring the students’ perception about the test. This study will fill the gap in the literature by providing students’ perception on the two tests method. Survey approach is conducted to obtain the data. The sample is collected in two identical classes with similar subject in a public university in Indonesia. Mann-Whitney U test used to analyse the data. The result indicates that there is a significant difference between two groups of students regarding CBT usage. Student with different test method prefers to have test other than what they were having. Further discussion and research implication is discussed in the paper.


Introduction
Information technology (IT) advancement has changed the way education delivered from lecturers to students. Electronic learning (e-Learning) was introduced as new way of delivering lecture notes, giving assignment and doing test. By using e-Learning student can have plenty benefits such as timely lecture notes, easiness in accessibility and flexibility in time of study [1,2]. Assessment is critical in measuring student's learning process achievement [3]. With the increasing number of students in recent years, education needs new ways of doing assessment for more efficient [4]. In doing so, Computer Based Test (CBT) is introduced. As part of learning conducted electronically (e-Learning), CBT has some advantages such as easiness in accessibility, flexibility, efficiency, and more consistent result than PPT [1,2,4].
Even though plenty benefits have been found by prior studies, some researcher doubt the superiority of CBT compare to more traditional Paper and Pencil Test (PPT). Prior studies have indicated that CBT provide a faster way of collecting test result than its counterparts but it did not have significantly different result [5,6]. In addition to that recent studies strengthened the argument by stating both test CBT and PPT have similar performance with notes on that computer administered test need more attention on the technical considerations [7] such as its acceptance by the intended users.  CBT adoption has been studied by various studies to seek the predictors of its success. The acceptance and success factor has been studied with various independent variable to predict e-Learning acceptance [8][9][10]. Meanwhile other studies had probing the instructor's perception and the complexity on the e-Learning system where CBT is included [11].
Having described some of the prior research, it is known that the debates of CBT were focused on the acceptance, performance and the complexity of the test itself. Prior studies rarely discuss about student perception toward both test types [12]. Therefore this preliminary study will fill the gap by providing insight on students' perception on both tests. Such understanding will help academics in providing a better test for students. Having that aim this study will seek the difference between two groups of students on their perception on CBT and PPT. In order to obtain the result, this study examined a hypothesis as follows: H0: There is no difference between students in perceiving CBT as tools of assessment

Methods
This paper is quantitative in nature. It used quantitative technique to describe student's perception towards CBT. Data collection in this study was using online-survey with close-ended questionnaire by asking students in a higher degree vocational education that is pursuing diploma certificate in administration science. Sample was taken from two classes: A and B. The classes were grouped as follows: A-class with PPT as the mean of final exam and B-class with CBT as the mean of final exam). Both classes have exactly the same subject that is computer application for administration science. All of the class members have had experience in using CBT at the mid exam of the same subject. All 30 members of students in each class were invited to fill the questionnaire. The questionnaire was developed based on prior studies regarding e-Learning use [13] with adjustment to the CBT context. The question that used in this preliminary study was "I prefer computer based test to paper-pencil test" using five point likert scale ranging from strongly disagree to strongly agree. After the data was collected statistical procedure was conducted to obtain the correct test. First Shapiro-Wilk tests of normality to test the nature of the data, followed with Levene's test for checking the homogeneity and ends with Mann-Whitney U to examine the difference between the two groups. Finally to enrich the meaning descriptive statistics was used to obtain the complete understanding about the two group difference.

Results and Discussion
The survey was conducted a month after the final exam was taken and the final score has been released, so it was not intervene the students' final marks in the subject. Average final scores of these two classes were relatively equal, i.e. 3.7 in 1 -4 scale. This survey was voluntary so students may or may not fill-in the survey on their convenience. Among 30 of students who were invited 22 students from A-class and 29 students from B-class were responded (see Table 1). The normality test using Shapiro-Wilk technique indicates non normal distribution for both A and B class with p-Value < .05. The result indicated that Mann-Whitney U test is the correct procedure for testing the difference between the two groups. The next step is testing the Homogeneity of the data using Levene's test as part of the Mann-Whitney U test procedure (see Table 2). The Levene's test indicate that the data variance is homogenous (p-Value=0.690 > 0.05). This result justified and strengthened the decision to use of Mann-Whitney U test.
The Mann-Whitney U Test presented in table 3 shown that the p-Value is less than the critical value (.03 < .05). This result indicates that the H0 should be rejected, meaning that there is significant difference between the two groups on the student's perception of the preferences of CBT. This finding is interesting to probe further, thus descriptive statistics was used for deeper understanding on how the agreements were differ. As mentioned before question about test preferences: "I prefer computer based test to paper-pencil test" was asked in the survey. The results are presented in figure 1 and 2.  Figure 1 shown that 59% of students that using PBT as their final exam prefers to use CBT as the means to conduct a test. Meanwhile only 14% of them were disagree of CBT usage for conducting a test. The other students have neutral opinion towards the CBT test (27%). It means that most of the students that used PPT prefers CBT as the tools for final assessment.   Figure 2. Diagram of B-class students' preferences on CBT. Figure 2 shown that 31% of students that using CBT as their final exam prefers to use CBT as the means to conduct a test. Meanwhile only 34% of them were disagree of CBT usage for conducting a test. The other students have neutral opinion towards the CBT test (35%). This result contradicts A-class opinion where students with PPT prefers CBT while those students who use CBT were mostly prefers PPT.

Class with PPT as Final Exam (A)
The hypothesis test shows that there was significant difference between two groups in this preliminary research. By having this result it can be stated that the student's perception on two modes of test: CBT and PPT is different. Such finding was then probed further by using descriptive statistics as the second step of analysis. The data presented in figure 1 and 2 indicated an interesting result, where most of students that used PPT as their final exam were more favouring CBT while on the contrary most of the students with CBT were not favouring CBT as the tools of assessment. This inconsistent finding is interesting. In the condition where both classes have experience in having CBT in the prior test and relatively have similar performance in their final exam, they have different perception toward the use of CBT.
Prior studies found that computer unfamiliarity as the reason of students' negative perception toward CBT [14,15]. However this finding might not relevant with the case in this study, where samples were taken from class with computer related subject. Therefore they must be familiar with computer usage. Moreover they have taken CBT on their mid-exam with most students have good result. Another studies found that social anxiety disorder (SAD) has hindered students in performing their best during an examination using CBT method [15]. It might be the reason why the students in Bclass did not favour CBT as the tool for final assessment. On the contrary students that having PPT as the way of doing final assessment might also having similar anxiety so they imagining of having CBT as their tool preference. This finding argues that most of those students having SAD, so they tend to imagine other forms other than what they were having.

Conclusions
This preliminary study result opposed prior studies that mostly conclude there was no difference between the paper pencil test (PPT) and computer based test (CBT) [16][17][18][19][20]. This might be based on the condition where most prior studies were primarily focused on the test performance whereas this study focused on the students' perception. Interestingly those students who have different perception were reversed, those who have PPT preferring CBT on the contrary those who were having CBT prefers other forms of test. This condition might be based on the presence of social anxiety disorder (SAD) as has been found by Tilfors [15]. The result strengthened prior studies [15] as well as completing the result on how other forms of test (PPT) also trigger SAD for the students. Practically, this result suggests that preparing student to control their anxiety before test is essential before examination, neither by using PPT or CBT. For further studies this research suggests the use of anxiety concept as the variable that influence students' perception towards CBT and PPT use.