Evaluation of Online Learning in Vocational High School Major in Accounting and Financial Institution: Kirkpatrick’s Model

. Online learning during the pandemic of COVID-19 in accounting and financial institution major of Public Vocational High School 1 of Boyolangu has not been evaluated as yet, which makes the effect and result of the learning still unknown; therefore, this research aimed to evaluate the online learning established at the school. The model of evaluation adopted in this research was Kirkpatrick’s two levels of evaluation: reaction and learning. The purposes are to measure students’ satisfaction with the online learning and understanding of subject materials given during the online learning. The data in this research were collected through questionnaires. The research found that students on the whole felt satisfied with the establishment of online learning during the pandemic for at least two reasons: students began to feel comfort and used to online learning and the information technology skills of both teachers and students improved by virtue of online learning. However, there were some problems which impaired the effectiveness of online learning, including limited internet quota and poor internet connection. Additionally, students had difficulty completing practical assignments because some of them did not possess laptops, and sometimes teachers had not given them any explanation of the subject materials before.


INTRODUCTION
The educational system in Indonesia has changed: students have to attend online learning meetings instead of face-to-face meetings due to the pandemic of a disease caused by particular viruses. To break the chain of the COVID-19 transmission, especially in school environment, the Minister of Education and Culture of the Republic of Indonesia issued the Learning-at-Home policy, which enclosed schools and universities, temporarily stopped face-to-face learning, and forced students to engage in distance learning, or online learning. Such policy has been implemented in some other countries to ensure the continuity of educational provision.
In the beginning, the pandemic of COVID-19 disrupted learning processes because both teachers and students faced obstacles in the way of establishing online learning. Students had to face such obstacles as students' deficiency in laptops for completing practical assignments on accounting, instable internet connection, limited internet quota, incomplete learning modules, and uninteresting learning sources and media. Meanwhile, the obstacles teachers had to face included instable internet connection and their lack of IT skills required to present interesting learning media to students. According to the result of interviews with somes students, the difficulties students had to deal with included difficulty in understanding subject materials in online learning, teachers' inadequate explanation, teachers' always giving assignment without having explained the subject, and limited internet quota for accessing subject materials.
According to Arsyad (2011) online learning, or usually referred to as e-learning, is a medium for supporting education, not for replacing classroom learning. The process of e-learning as a medium for distance learning has established a new paradigm, in which teachers play a role as facilitators and students serve as active participants in the teaching and learning process. Therefore, teachers are required to take part actively in learning processes. The use of online learning system is one of the efforts to be made to settle educational issues and ease students' burden of accessing subject materials. Riyanda, Herlina, and Wicaksono (2020) explained that the important thing to do during online learning sessions is to forge online communication and discussion with fellow students. Sutarsih and Kadarsih (2007) explained that evaluation is an activity of measuring a value of an object in a particular parameter. Evaluation is the first stage in an assessment process, in which assessment is one of the aspects of educational evaluation, and evaluation is the process of assessing the overall academic programs (Komalasari, 2013). Furthermore, Yumarlin (2016) also explained that evaluation is in general a process of providing information about how far an activity has achieved its target. Farida (2017) stated that evaluation is a systematic, continuous, and comprehensive program in order to provide quality control, assurance, and standards for all the components of learning process under further consideration and according to particular criteria. In other words, evaluation is a decisionmaking process on the basis of the result of assessments, on which the enforcement of a policy is based.
The evaluation of learning program is to make an estimate of learning implementation to examine its effectiveness and progress in order to achieve the intended learning purpose (Soetopo, 2007). Arikunto (2014) explained that the result of an evaluation may take a form of an evaluator's recommendation for decision-making processes. The result of program implementation may be the basis for four possible decisions: 1) to scrap a program, because the program produces no benefit, or because its implementation is below expectation; 2) to revise a program, because a few of its parts are below expectation; 3) to continue a program, because the implementation of the program manages to satisfy the expectation and produces beneficial results; and 4) expand a program, because the program achieves a considerable success and it will be better if the program is launched in other places ant time.
There are many approaches to be adopted to evaluate a program. Fitzpatrick, Sanders, and Worthen (2011) explained that the difference in approaches to evaluation arises from the backgrounds, experience, and worldview of the writer. One of the evaluation models is Kirkpatrick's evaluation model, which was designed to evaluate programs. This evaluation model consists of four levels, namely reaction, learning, behavior, and result. Kirkpatrick and Kirkpatrick (2006) stated that reaction evaluation measures the level of satisfaction with the program accomplishment; learning evaluation measures what has been achieved and proved; behavior evaluation measures trainees' behavioral changes after training programs; and result evaluation measures the final result with the focus on the advantage for the institution.

METHODS
This was evaluation research using the quantitative, descriptive approach. The evaluation model used was Kirkpatrick's level one (reaction) and Kirkpatrick's level two (learning). The research instrument used was questionnaires. The research involved 180 respondents as the research sample distributed through questionnaires. The mastery of C1 5 The mastery of C2 3 The mastery of C3 2 The questionnaire in this research asked closed-ended questions with the indicators shown in Table 1. The questionnaire of this research was developed using Likert scales. The reason why Likert scales were used in this research was the fact that those scales could be used to measure the behavior, opinion, and perception of individuals or groups of individuals of social events or phenomena (Riduwan, 2013). The measurement model used was Likert scales with four response categories. The alternative responses included very satisfied (vs), satisfied (s), dissatisfied (ds), very dissatisfied (vds). Subsequently, the data collected were analyzed using the quantitative, descriptive analysis techniques.

RESULTS AND DISCUSSIONS Kirkpatrick's level 1 (reaction)
Kirkpatrick's evaluation model level 1 (reaction) is used to examine students' response to the implementation of online learning and to measure students' level of satisfaction with online learning with all obstacles during the implementation of online learning.
180 respondents of the eleventh and twelfth graders answered question items about variables in the evaluation of online learning using Kirkpatrick's level one (reaction). The detailed explanation of the percentage of respondents' response in relation to the variables of the evaluation of online learning using Kirkpatrick's level one (reaction) is shown in Figure 1.  Figure 1, 11 students expressed great satisfaction (6%), 169 students expressed satisfaction (94%), and no student expressed less satisfaction and dissatisfaction (0%). This means that by using the evaluation model of Kirkpatrick's level one (reaction), students felt satisfied or agreed with the online learning, only that in the choice of online learning and face-to-face learning, students preferred the later to the former because students felt it is easier to build up closer relationship with fellow students or with teachers and ensure effective coordination in completing assignments.

Kirkpatrick's level two (learning)
Kirkpatrick's level two (learning) is aimed at assessing the quality of materials received by students majoring in Accounting and Financial Institution with KD.3.10 by recording a transaction on a specific journal of a trading company. 180 respondents answered question items about variables in the evaluation of online learning using Kirkpatrick's level two (learning). The detailed explanation of Kirkpatrick's level two (learning) is shown in Figure 2.  Figure 2, using the evaluation model of Kirkpatrick's level two (learning), it was known that 63 of 180 respondents were very satisfied (35%); 115 respondents were satisfied (64%); no respondent was less satisfied (0%); and 2 respondents were dissatisfied (1%). Therefore, by implementing the evaluation model of Kirkpatrick's level two (learning) to the students majoring in Accounting and Financial Institution of Public Vocational High School 1 of Boyolangu, it could be concluded that students felt satisfied or agreed with what has been learned.
However, the evaluation model of Kirkpatrick's level two (learning) is difficult to be implemented in the subject containing practical assignments. Besides, given that not all students possessed laptops, materials delivered during online learning was only at the level of knowledge (C1) and understanding (C2). In this case, subject materials containing practical assignments should have been delivered in such face-to-face learning that students could master the lesson to the level of practical application (C3) (Bloom, 1956).

CONCLUSION
In general, students majoring in accounting and financial institution at Public Vocational High School 1 of Boyolangu were satisfied with the learning program they were receiving. They felt comfortable because they managed to understand how to easily access the link for learning. They had no longer difficulty finding location in which to find good internet connection during the pandemic. With the help of the government giving them internet quota for free, they had just to manage their time: some time for learning and some other time for doing online business.