Determining priority criteria for using online learning media in Emergency Remote Teaching (ERT) using the Analytical Hierarchy Process (AHP) method

Many institutions in Indonesia swiftly shifted their educational programs to online learning with minimal preparation due to the COVID-19 pandemic. This has resulted in a situation termed as Emergency Remote Teaching (ERT), defined as a temporary and abrupt shift from offline (face-to-face) to online (virtual) instructional delivery as a consequence of a disaster. It is crucial for educators to grasp key aspects of utilizing online learning media. This research aims to identify recommended criteria and alternatives for the use of online learning media from the educators’ perspective, employing the Analytical Hierarchy Process (AHP) method, which is suitable for ranking issues. This method involves a group decision-making process that employs comprehensive and multidimensional ranking in various domains. The research instrument comprises a questionnaire filled out by educators, including lecturers and teachers, totaling 13 respondents. The study reveals that the top-ranked criteria and alternative for the use of online learning media are the ease of operation criterion (K1) and the fastness alternative (A1). Future research is encouraged to further explore the application of the AHP method in other aspects of online learning.


Introduction
Prior to the COVID-19 pandemic, E-learning experienced an annual growth of approximately 15.4% in educational institutions worldwide, devoid of uncertainties or pressures on institutions or learners (Dao Thi Thu & Duong Hong, 2021).Nevertheless, the circumstances have undergone significant transformation during the era of COVID-19.Due to the global restrictions aimed at mitigating the spread of COVID-19, educational institutions have shifted a substantial portion of their services online, encompassing instructors and various assessments, to over 60% of learners worldwide (UNESCO, 2020).The COVID-19 pandemic has had implications on various governmental institutions, particularly in the realm of education.Many educational establishments swiftly transitioned their educational programs to online distance learning with minimal preparation (Kristanto et al, 2020).As a result, educational institutions were compelled to cancel in-person learning as a measure to mitigate the risks posed by this virus (Saputra & Rusmana, 2021).This has led to a situation known as Emergency Remote Teaching (ERT), which is defined as a sudden temporary shift from in-person (offline) to online (virtual) learning due to the impacts caused by a disaster, distinct from pre-planned virtual online courses (Hodges et al, 2020).On the other hand, educators are expected to serve as mediators and facilitators in the ERT process, going beyond mere replication of face-to-face instruction in online learning, and enabling learners to take a more active role in their own educational journey (Yildirim, 2020).Regardless of the distinctions between distance education and ERT, instructors play a crucial role in the effective teaching process.This is evident from the emphasis on 'teaching' in the ERT process undertaken by educators (Hodges et al, 2020).
The utilization of online learning media can serve as a solution to ensure a seamless learning process (Aswir, & Rosiana, 2021).Virtual meetings conducted through online media offer several advantages, particularly video conferences, which enhance learning by making it effective, convenient, and secure (Pratama et al, 2020).Multi-Criteria Decision Making (MCDM) involves selecting the best alternative in making robust and complex decisions.MCDM aids in choosing the optimal alternative from numerous criteria that can be obtained by analyzing the scope of criteria, weighting them, and selecting an optimal outcome using multi-criteria decision-making techniques (Dalalah et al, 2010).The Analytical Hierarchy Process (AHP) is a technique within MCDM that is suitable for ranking issues (Cabała, 2010).The AHP is a decision support method developed by Thomas L. Saaty for group decision-making that employs a widely applicable ranking process across various domains (Aziz et al, 2016).In AHP, factors that can influence decisions are identified and structured hierarchically to reduce the complexity of decision problems across various levels, and then appropriate levels are prioritized using Pairwise Comparison (Pant et al, 2022).Hierarchy is defined as a multi-level structure designed to represent complex problems with multiple levels (Saaty, 2008).The study examines the factors utilized to evaluate the quality of online learning in higher education stemming from the COVID-19 pandemic (Cho, & Woo, 2022).In Mu's study, et al. (Mu et all, 2022), the aim was to identify and measure the primary challenges faced by students, such as inadequate facilities, difficulties with learning media, and financial constraints.As mentioned in the previously described research with specific objectives, one of which pertains to online learning media.Hence, this study is centered around the utilization of online learning media.
In the context of ERT, AHP can be employed to establish priorities for the most suitable use of online learning media based on criteria and alternatives.This media faces limitations across several factors, including social, economic, and environmental aspects.AHP is not only used to select the utilization of online learning media but can also be employed to evaluate the effectiveness of the utilized online learning media.By defining appropriate criteria and alternatives, AHP can be employed to assess the effectiveness of online learning media.Considering the objectives and problem constraints previously outlined, this research is aimed at determining criteria and alternatives related to the utilization of online learning media.It seeks to ascertain the ranking results of online learning media usage from an educator's perspective and aims to provide valuable insights for educational evaluation in Indonesia.

Design of Research
The purpose of this study is threefold: first, to establish a method for determining criteria and alternatives for online learning media; second, to derive criteria and alternatives relevant to the use of online learning media; and third, to provide recommendations regarding the utilization of online learning media from an educator's perspective.The Analytical Hierarchy Process (AHP) is employed to determine the criteria and alternatives for online learning media usage.This method is used to ensure logical consistency in evaluations aimed at establishing priorities (Saaty, 1990).

Participants in Research
The participants involved in this research are educators such as lecturers and teachers who have previously utilized online learning media during the Emergency Remote Teaching (ERT) period.The study comprises a total of 13 participants.It employs primary data.The data collection method employed is questionnaire-based.Questionnaires were distributed to the participants of this study.
The selection of criteria and alternatives for this research is drawn from the research study conducted by Wang (Wang et al, 2021).

Research Instrument
This study employs a research instrument in the form of a questionnaire or survey to determine criteria and alternatives for the usage of online learning media.The questionnaire is targeted towards qualified respondents, namely educators who have conducted teaching through online learning media during the Emergency Remote Teaching (ERT) period.The questionnaire consists of multiple questions concerning the importance assessment, utilizing a pairwise comparison scale, regarding the criteria and alternatives for the utilization of online learning media as conceptualized (Saaty, 2001).

Analysis of Research Data
This research employs the Analytical Hierarchy Process (AHP) as the data analysis method.The AHP method is characterized by several main principles, including:

Decomposition
The process of decomposition involves breaking down a complex system into smaller, understandable components.In this research, the multicriteria within AHP is simplified into a hierarchical structure comprising objectives (first level), criteria (second level), and alternatives (third level).

Comparative Judgement
The pairwise comparison process is a method of comparing the importance level between two criteria or two alternatives based on decision-maker assessments.In this research, the hierarchical system is structured as an n x n matrix, presented using a pairwise comparison matrix table as shown in Table 1.The two elements are equally important, with the same influence.3 One element is slightly more important than the other.5 One element is more important than the other.7 One element is significantly more important than the other.9 One element is absolutely more important than the other.2,4,6,8 A value between two adjacent preference values.

Synthesis of Priority
The Geometric Mean (GM) is a step performed prior to the priority synthesis process, involving the calculation of the average value from the pairwise comparison assessments across all respondents.The calculation is as follows: The priority synthesis process involves determining the contribution magnitude of each criterion.The steps in this process include summing the values of each column in the matrix, then dividing each value within a column by the total of that column to obtain a normalized matrix.The subsequent step is summing the values within each row and dividing by the number of elements to derive an average value.

Calculating the Vector [X]
Vector

Logical Consistency
The logical consistency process is a stage in decision-making to assess the consistency level of the data.In this phase, it involves calculating the Consistency Index (CI) and the Consistency Ratio (CR) using the following formulas: The values for the random index are indicated in Table 3. Table 3 presents the random index values, which consist of n representing the number of elements and RI denoting the random index.If the consistency ratio is less than or equal to 0.1, the calculation is considered consistent.Conversely, if the consistency ratio exceeds 0.1, the data assessment needs to be revised.

Rangking Weights
After normalization to obtain the vector   = ( 1 ,  2 , . . .,   ).Take weight  1 ,  2 ,  3 ,   4 from the alternative matrix as a column vector to form matrix D, and multiply it by weight vector   to obtain ranking weights, as follows: B = D x BK (7) B = Rangking weights D = Alternative weight matrix   = Criterion weight vector

Results
The criteria and alternatives for the usage of online learning media were obtained from a journal, comprising four criteria with codes K1 to K4.Each criterion consists of four alternatives, resulting in a total of 16 alternatives labeled from A1 to A16 (Wang et al, 2021).The criteria and alternatives for the usage of online learning media are presented in Table 4.The presentation of data in the form of a hierarchy diagram of criteria and alternatives for the usage of online learning media is depicted in Figure 1.The criterion has an λmax = 4.127, CI = 0.042, CR = 0.047 < 0.1, indicating consistent weighting.Alternatives for K1, 'Ease of Operation' have λmax = 4.182, CI = 0.061, CR = 0.068 < 0.1; alternatives for K2, 'Functional Completeness' have λmax = 4.029, CI = 0.010, CR = 0.011 < 0.1; alternatives for K3, 'Interface' have λmax = 4.138, CI = 0.046, CR = 0.051 < 0.1; alternatives for K4, 'Technical Excellence' have λmax = 4.229, CI = 0.076, CR = 0.085 < 0.1.All alternatives with respect to criteria exhibit consistent weights.

Discussion
Determining online learning media involves assessing various factors that contribute to the success of learning outcomes in the digital educational environment (Ertmer et al, 2011).Online learning media has become an effective educational method, especially in the digital age where technology is easily accessible to a large population (Collins & Halverson, 2018).In this study, determining criteria and alternatives aims to identify the prioritized aspects in using online learning media for educators who utilize online learning resources to support online teaching.As evident in Table 1, the criteria are labeled as K1 to K4, and the alternatives are labeled as A1 to A16.This builds upon previous research (Wang et al, 2021).indicates that the criteria 'Ease of Operation' and the alternative 'Fastness' are given higher priority in the utilization of online learning media.In terms of criteria, they are followed by 'Technical Excellence', 'Interface', and 'Functional Completeness'.Meanwhile, in terms of alternatives, they are followed by 'Stability', 'Versatility', 'Safety', 'Fault Tolerance', 'Compatibility', 'Clarity', 'Freedom', 'Teaching Resource Accessibility', 'Show Diversity', 'Feedback', 'Reusability', 'Aesthetic', 'Teaching Presence', 'Consistency', and 'Data Visibility'.Another study's findings (Wang & Lin, 2019) corroborate the same result, where 'Ease of Operation' ranks as one of the top priorities or the prioritized aspect concerning the usage of online learning media.
In the study by Wang (Wang et al, 2021) the results align with the current researcher's study, where 'Ease of Operation' criterion and the 'Fastness' alternative share the top rank.However, the subsequent rankings for both criteria and alternatives differ between the two studies.This study has certain limitations to consider, such as its focus solely on educators like teachers and lecturers who have used online learning media as users, and the lack of emphasis from the learner's perspective.In terms of determination, the research findings recommend prioritizing 'Ease of Operation,' 'Functional Completeness, ' 'Technical Excellence,' and 'Interface' as criteria; and 'Fastness,' 'Feedback,' 'Versatility,' as well as other factors as primary factors in selecting the usage of online learning media.This offers guidance to educators in highlighting key aspects when choosing online learning media.

Conclusion
Based on this research, it can be concluded that the determination of criteria for the usage of online learning media during the Emergency Remote Teaching (ERT) period, using the Analytical Hierarchy Process (AHP) method, resulted in criterion and alternative CR values that are less than 0.1, indicating consistent prioritization weights for both criteria and alternatives.The recommendation for criteria and alternatives for the usage of online learning media from the educator's perspective has been established, wherein the prioritization weights are ranked with K1 (Ease of Operation) taking the first place, followed by K2, K4, and K3.The top-ranked alternative is A1 (Fastness), followed by A3, A2, A7, A5, A6, A13, A9, A4, A8, A14, A11, A10, A16, A15, and A12.

Fig 1 .
Fig 1. Hierarchy of Criteria and Alternatives ), resulting in the values below:

Fig 2 .
Fig 2. Criterion and Alternative Ranking Weights

Table 1 .
Pairwise Comparison Matrix

Table 1
displays the pairwise comparison matrix designed to represent the relative importance of one element compared to others.In establishing the decision-maker's scale, this research employs the pairwise comparison scale set by Saaty, as illustrated in Table2.

Table 2 .
Pairwise Comparison Scale

Table 4 .
Criteria and Alternative

Table 5 .
Pairwise Comparison Matrix of Criteria

Table 6 .
Pairwise Comparison Matrix of Alternatives with respect to K1

Table 7 .
Pairwise Comparison Matrix of Alternatives with respect to K2

Table 8 .
Pairwise Comparison Matrix of Alternatives with respect to K3

Table 9 .
Pairwise Comparison Matrix of Alternatives with respect to K4