Evaluating e-learning systems success in the new normal

The goal of this study is to develop and verify a model for successful e-Learning based on the expe- riences of students in the "new normal". From Jordanian universities, 550 students who have taken any e-Learning course were randomly selected. Data were collected via a survey questionnaire, and Structural Equation Modeling (SEM) was employed to test the proposed study model. The findings indicate that contactless learning and high-quality e-learning systems have a beneficial impact on student satisfaction. In addition, e-Learning cognitive involvement was found to solidify e-Learning satisfaction. Furthermore, the results show a positive and significant impact of e-Learning cognitive involvement and e-Learning satisfaction on e-Learning achievement. Also, e-Learning system quality positively affects e-Learning cognitive involvement, besides a direct impact of contactless learn- ing quality on e-Learning cognitive involvement.


Introduction
Information and Communication Technology (ICT) advances have changed the way people deal with knowledge. For instance, ICT today allows the sharing of information unbounded by time and place (Martınez-Torres et al., 2008;Dron & Anderson, 2022). COVID-19 pandemic outbreak that began in 2019 had dramatically altered the education setting, whereby e-Learning became the primary way of learning in educational institutions Heo et al., 2022). Additionally, the social and economic activities were carried out in contactless environments -all these have become the new normal. Elnaj (2021) described the new normal as changes that have occurred in the behavior of people during or after the pandemic. Owing to the pandemic, many education institutions (if not all) all over the world were forced to halt their normal educational activities. For most of these institutions, e-Learning was the chosen form of learning offered to students. E-Learning, which was first introduced in the 1960s as "Computer-Assisted Instruction" (Anderson, 2008), involves technology application in forming educational experiences, and has since evolved. As a contactless digital technology (Hamid, 2001;Downes, 2005;Ebner, 2007;Adedoyin & Soykan, 2020;Almaiah et al., 2020), e-Learning is an appropriate form of learning during the pandemic. Equally, e-Learning has been viewed as a shift from the more conventional approaches to education (Allen & Seaman, 2008;Cheng, 2012;Al-Bashayreh et al., 2022). Farhan et al. (2019) stated that the integration of social network services, online forums and real-time interactive systems might motivate participation of learners in e-Learning. In universities, e-Learning has been effective in the delivery of courses to students (Laurillard, 2004). In a business context, the performance of employees could be increased with the use of technology-based educational tools (Bates & Bates, 2005).
In higher education, e-Learning comes in two forms namely Massive Online Open Courses (MOOCs) and online universities, with the former one being the more popular option for offering high-quality educational systems worldwide. The popularity of MOOCs has been mainly factored by its ability in fulfilling the needs of occupied individuals in expanding their skills and knowledge in certain domains. The latter type, namely online universities, facilitate the distance and open learning programs for those who wish to complete their degrees online. Some universities offer all their courses online, for instance, Massachusetts Institute of Technology in the United States, in 2007.
Undeniably, e-Learning applications have been increasing, and yet, e-Learning success has not been adequately examined in terms of its concept and elements. E-Learning has been shown to be ineffective cost-wise (Horton, 2001;Nagy, 2005;Nicholson, 2007;Mohammadyari & Singh, 2015;Islam, 2016), and have lower substitution effect on traditional learning systems. The satisfaction of users towards this form of learning has been reported to be lower as well (Packham et al., 2004). E-Learning systems require active participation of learners, otherwise, the systems may not be successful -success is dictated by learner's learning performance. Studies including Contini et al. (2018) and Sun et al. (2008) have reported that only a few that enrolled in e-Learning courses had passed their courses. Furthermore, in 2017-2018, the completion rate of e-Learning courses through MOOCs was very low, at 3.13% (Lederman, 2019), leading to undesirable e-Learning experiences not only for students, but for instructors as well. Based on that, success of e-Learning means high completion rates in fully online taught courses.
In addition, the extant e-Learning studies did not look into the many success aspects of e-Learning. In fact, in most of these studies, success was employed as a sole element of either the system (Packham et al., 2004;Wang et al., 2007), the student (Chen & Liu, 2013), or the lecturer (Oliver & Herrington, 2003;Bacca et al., 2014). Task, people, roles and technology are the four constituents of an information system (O'Hara et al., 1999). Specifically, role tasks are carried out with the use of technologies, while roles (or structure) are associated with "the communications, authority and workflow systems within the organization" (O'Hara et al., 1999, p. 64). Within the setting of e-Learning, the experiences of students comprise the combination of the aforementioned four constituents. As an illustration: Side by side, teacher and student denote people and role in e-Learning context. Utilizing e-Learning systems, teachers carry out lecture sessions to students; e-Learning tasks refer to lectures, while technology refers to the e-Learning system.
The model used in this study was the Information Systems (IS) success model by DeLone and McLean (2003). This model was appropriate for its holistic view of the e-Learning system. The model also covers the many aspects associated with the achievement of e-Learning namely learning system, student and lecture. Also, it describes the main dimensions of these aspects and the relationships between them, contributing to the success of IS. As mentioned, e-Learning essentially involves the interactions between students and lecturers via a learning system, and so, its success may require cooperation of some important dimensions. For this reason, the IS success model could comprehensively examine e-Learning success. In fact, this model has been employed in past studies on e-Learning success (Freeze et al., 2019;Al-Fraihat et al., 2020;Subaeki et al., 2020;Shahzad et al., 2021), the exploration was not as comprehensive. In this study, the model was used in examining e-Learning success in the new normal, utilizing the e-learning experiences of students as one of the factors. Accordingly, the key question to address was: What are the students' experiential factors affecting e-Learning success?

E-Learning success
E-Learning is a web-based system that provides information or knowledge to users, without geographical or time restrictions. Relevantly, e-Learning success refers to the degree to which a student, as an e-Learning system evaluator, is confident that learning a course through e-Learning is sensible (Kim et al., 2003). In e-Learning system evaluation, its components and their relationships need to be evaluated from a comprehensive outlook. Accordingly, students, teachers, lectures and systems are the key constituents of e-Learning, and so: a student enrolls on a course which involves participation, assignment completions, and so forth, through an e-Learning system, while a teacher, via the e-Learning system, presents to students teaching materials via giving lectures and via answering the questions posted by students.
E-Learning has been proven to be more advantageous compared to conventional face-to-face education (Piccoli et al., 2001). However, the implementation of e-Learning has its own challenges as it can be time consuming and laborious. In addition, e-Learning environments require specific material resources. In their study, Arbaugh and Duray (2002) reported that the implementation of e-Learning courses was not only costly, but their completion rate was low as well, and this issue needs to be addressed by both management and system designers. Notably, e-Learning success has been examined in many educations and IS studies, utilizing technology related models including Technology Acceptance Model (TAM) (Ajzen & Fishbein, 1977;Davis, 1989) and expectation and confirmation model (Bhattacherjee, 2001;Wu et al., 2006). Shahzad et al. (2021) and Subaeki et al., 2020) were among those who have examined e-Learning systems utilizing a single method like an experiment or a questionnaire. Meanwhile, in identifying the factors affecting e-Learning usage, Mohammadi (2015) employed a combination of TAM and IS success model, but the author was only examining e-Learning usage, not e-Learning success from the context of the experiences of students, and the factors addressed in this study were all related to the system and the student. Success of e-Learning in terms of the learning system, lecture, teacher and students also has not been adequately examined. Hence, there was a gap in the literature that needed to be addressed. E-Learning in the new normal involves two major components of learning and technology, whereby the former involves knowledge acquisition or modification facilitated by technology (Aparicio et al., 2016). In higher education, various e-Learning modalities have emerged, with MOOCs and online universities currently being the two major ones. MOOCs and online universities cater to learners who cannot be present owing to geographic, economic or political restrictions (Aparicio et al., 2016). Past studies on e-Learning were mostly focusing on its course content and activities (Brox et al., 2004). On the other hand, the more current ones were focusing on e-Learning systems (Lee et al., 2005;Chen & Liu, 2013), while others were looking at perceived satisfaction of learners towards e-Learning (Sun et al., 2008;Aggelidis & Chatzoglou, 2012). Sun et al. (2008) for instance, reported seven major factors impacting perceived satisfaction of learners as follows: flexibility of e-Learning course, attitudes of instructors towards e-Learning, e-Learning course quality, computer anxiety, perceived usefulness, perceived ease of use and evaluation diversity. Following the outbreak of COVID-19, e-Learning studies within the context of new normal began to emerge, and many were focusing on the role played by technology and the universalization of contactless education services. Furthermore, the increased use of e-Learning has led to the interest towards how e-Learning systems can technically facilitate both instructors and learners (Bozkurt & Sharma, 2020;Elhaty et al., 2020;Pham & Ho, 2020;Rashid & Yadav, 2020;Muller et al., 2021). Equally, e-Learning usage based on country in the wake of the pandemic is worthy of exploration as well (Tirziu & Vrabie, 2015;Tria, 2020;Alhumaid et al., 2020;Naddeo et al., 2021;Phuthong, 2021). Despite the increase in e-Learning use, its success factors are still indefinite. However, some studies have mentioned the importance of user participation, adoption and satisfaction and their antecedents in determining e-Learning success. Somehow, the essential factors for success are still unknown.

IS success model and integrated framework
In construing the factors obtained from the exploratory study and linking them with the IS success model, IS success model was used in this study. Freeze et al. (2019) relevantly stated that system quality in e-Learning can be demonstrated through its aptness towards its intended use and users' needs, while information quality relates to e-Learning content. From the perspective of education, service quality of e-Learning relates to the quality of knowledge delivered by instructors to learners. In other words, it concerns the skills of instructors and online interactions in the delivery of the course. As posited by IS success model, system quality, information quality and service quality impact e-Learning use and satisfaction of users. Additionally, the model proposed use and users' satisfaction as direct determinants of net benefit, which can be used in measuring the individual outcomes of e-Learning like learning performance. This study proposed an integrated framework based on the IS success model. The purpose of the framework was to identify the critical factors for e-Learning success. Accordingly, from the e-Learning perspective, five constructs belonging to the four dimensions of the IS success model (i.e., system quality, information quality, service quality and net benefits) were identified. Specifically, System quality pertains to the sought-after IS system characteristics (e.g., availability and usability). Accordingly, system quality in e-Learning is measurable via the e-Learning system characteristics. Additionally, to serve the study purpose, two constructs namely e-Learning cognitive involvement and e-Learning satisfaction were added to the model.

Research model and hypotheses
An e-Learning success model was proposed in this study, as shown in Fig. 1; the model was based on the integrated framework that comprise four constructs affecting use and user satisfaction: system quality expressed as e-Learning system quality; information quality expressed as perceived fit of e-Learning content; service quality expressed as e-Learning instructor quality; and student-instructor online interactivity. In IS success model, the constructs of instructor quality and online interaction are not essentially exclusive (DeLone & McLean, 2003), and so, the construct of contactless learning quality was included as a second-order construct, comprising the dimensions of e-Learning instructor quality and student-instructor online interactivity. Meanwhile, the use construct expressed by e-Learning cognitive involvement, impacts user satisfaction expressed as e-Learning satisfaction. There were five control variables included in the research model namely e-learning system quality and contactless learning quality. Finally, the construct of net benefit, expressed as e-Learning achievement, is affected by e-Learning cognitive involvement and e-Learning satisfaction.

Hypotheses development E-Learning system quality and e-Learning cognitive involvement and e-Learning satisfaction
System quality as the IS success model's important dimension, impacts IS use success (DeLone & McLean, 2003). Accordingly, Gorla et al. (2010) described e-Learning system quality as the degree to which the system's technical aspect sustains e-Learning. Within e-Learning context, website interface, audio quality and video quality can represent system quality (Novak & Hoffman, 1997). In their study, Weaver et al. (2013), found that the involvement of students in e-Learning systems is affected by website interface quality, and the available support structure. Higher-quality learning system requires e-Learning cognitive involvement of higher levels. Wixom and Todd (2005) found that the beliefs of e-Learning system quality form the attitudes towards the satisfaction towards both the system and information. Additionally, Ho et al. (2010) and Rai et al. (2002) reported positive effects of system quality on user satisfaction. Based on the experience of students, their contacts with e-Learning are primarily through the system, and so, students cannot experience e-Learning without the technical system itself. In other words, an increase in e-Learning system quality may increase satisfaction and involvement of learners. Therefore, two hypotheses were proposed: H1a: E-Learning system quality has a positive effect on e-Learning cognitive involvement.
H1b: E-Learning system quality has a positive effect on e-Learning satisfaction.

E-Learning cognitive involvement, e-Learning satisfaction, and e-Learning achievement
The construct of involvement has been used in examining stimulus objects like consumer satisfaction and purchase intention (Beldona et al., 2005). Within the context of learning, Tinto (1987) indicated that involvement of students promotes their effort and increases learning. Huang and Chang (2004) added that for students, higher levels of academic involvement will increase the number of benefits gained from learning and personal development. Within the context of marketing, Beldona et al. (2005) indicated that involvement and satisfaction determine consumer's purchasing behavior. For consumers of e-Learning (i.e., students), their main goal is to attain high-level academic achievement (Lee & Lee, 2008). Hence, success in e-Learning should include achievement. Meanwhile, e-Learning cognitive involvement concerns the perceived relevance of an e-Learning course according to interest in thinking about and learning information relevant to an offering (Jiang et al., 2010). Further, satisfaction of e-Learning concerns students' favorability level of their subjective evaluation of the e-Learning experience (Gu & Wang, 2015). Additionally, Lee and Lee (2008) described e-Learning achievement as the level of a student's perceived accomplishment in e-Learning. Hence, in contemplating the relationships between e-Learning cognitive involvement, e-Learning satisfaction and e-Learning achievement, the hypotheses below were proposed: H2: E-Learning cognitive involvement has a positive effect on e-Learning satisfaction.
H3: E-Learning cognitive involvement has a positive effect on e-Learning achievement.
H4: E-Learning satisfaction has a positive effect on e-Learning achievement.

Contactless learning quality and e-Learning cognitive involvement and e-Learning satisfaction
Service quality in IS success model suggests that customers need support from providers (DeLone & McLean, 2003). Meanwhile, Byrne and Flood (2003) mentioned quality teaching as the key objective of higher education, and it involves creation of pleasant programs, provision of feedback to students, elucidation of issues and concepts, inspiring students and demonstration of understanding towards students' problems. Teaching quality can be achieved through high-quality instructors and highquality interaction between instructor and students. Somehow, e-Learning occurs without face-to-face contact between instructor and student(s), and this is in fact a notable feature of this type of learning. Lack of face-to-face contact in e-Learning could impede quality teaching, and for this reason, Owlia and Aspinwall (1996) reported the need to focus on knowledgeability. On the other hand, scholars including Fauth et al. (2014) and Praetorius et al. (2018) have associated instructional success with the competency of instructors in establishing a learning atmosphere and in forming good relationships with students. Contactless learning quality concerns how far certain online activities promote student learning well, and several measurement methods and dimensions methods have been employed in evaluating this construct (contactless learning quality). Two dimensions were employed in this study in examining contactless learning quality namely e-Learning instructor quality and student-instructor online interactivity. As explained by Bruhn et al. (2008), both dimensions signify a learning quality aspect that could be a distinct but essential to contactless learning quality at a more abstract level. Meanwhile, Byrne and Flood (2003) described e-Learning instructor quality as the level of wholeness of teaching of an instructor of an e-Learning course. It also denotes the competency of the instructor. In higher education, students are the main customers, and therefore, teaching them becomes a main service and function of e-Learning (Hill, 1995). Sun et al. (2008) mentioned this construct as an important factor in learning effects and e-Learning satisfaction. Additionally, student-instructor and student-student interactions add to the outcomes of learning, and therefore need to be evaluated (Dwyer et al., 2004). Relevantly, Sher (2009) examined student-instructor online interactivity and described it as an interactivity between learner and instructor within a course of e-Learning. In fact, student-instructor interactions are vital in e-Learning and happen more regularly when compared to the conventional face-to-face class. An instructor is responsible for managing the e-Learning students, and is obliged to cater to the needs of students in their e-Learning journey, by providing appropriate responses and by helping them. Direct messages, online forums, comments, posts, and e-mail are among the tools through which student-instructor interactivity occurs. Jin (2005) accordingly stated that interactive learning increases students' involvement in online class discussions. Additionally, the effectiveness of web-based learning demonstrates the importance of interaction in the success of courses. Jin (2005) found that a highly interactive environment increases students' joy in learning, and so, increased interactivity may increase satisfaction and participation of learners. In universities, Biggs (2011) indicated that practical teaching and quality learning increases involvement of students in the courses that they'd enrolled in. Teaching, as an e-Learning component, has been linked to the two main actors of e-Learning, namely, instructor and student. Contactless learning quality is crucial in e-Learning success as it reflects the capabilities of instructors and also their relationships with students. Hence, this study proposed the following hypotheses: H5a: Contactless learning quality has a positive effect on e-Learning cognitive involvement.
H5b: Contactless learning quality has a positive effect on e-Learning satisfaction.

Sample size and data collection
550 students who had previously participated in e-Learning (i.e., who had taken any e-Learning course) were chosen at random from Jordanian universities and polled during the fall semester of 2021-2022. SEM was used to assess the collected data. The findings are then presented and examined to determine whether the factors used have any bearing on the degree of e-learning system success among particular users.

Respondents' demographic profile
According to the collected data, the majority of respondents (54.6%) were female, the majority (54.5%) were aged 30 or older, and the highest distribution of education was indicated by 200 respondents (36.3%); last but not least, 54.5 percent of the registrants said they had only attended four online courses. Table 1 displays the demographic profiles of the study respondents.

Measures
The questionnaire items were from past studies, with several modifications. All items were furnished with a 5-point Likert scale except for those in the demographic section. Specifically: Five items represented e-Learning system quality and these items were obtained from Kim et al. (2022); nine items represented contactless learning quality and these items were obtained from (Sher, 2009); five items represented e-Learning cognitive innovation and these items were obtained from Jiang et al. (2010); four items represented e-Learning satisfaction and these items were obtained from Chiu and Chen (2005); and four items represented e-Learning achievement and these items were obtained from Kim et al. (2022). As mentioned, the items were modified as needed to fit the study context.

Data analysis
Results of factor loadings, Cronbach's alpha, Composite Reliability (CR), Average Variance Extracted (AVE), and Heterotrait-Monotrait (HTMT) can be observed in Table 2. As shown, the loading and Cronbach's alpha values were within the 0.80-0.90 range as recommended; CR values were above the recommended value of 0.70 (0.94-0.81), while AVE values were greater than the cut-off value of 0.50 (Hair et al., 2013;Hair et al., 2014) (0.65-0.81). All these values affirmed the reliability and validity of the first-order constructs. Meanwhile, the Fornell-Larcker's (1981) criterion evaluation results are shown in Table 3. The results in both Table 2 and Table 3 show that all the obtained values fulfilled the recommended criteria. Hence, the study's measurement model is reliable as shown in Table 4.

Assessment of Measurement Model
Maximum Probability (ML) is an estimation method commonly employed in concurrent model parameters estimation, and ML is appropriate for small sample sizes (100 to 200), and so, was appropriate for this study's dataset. This study employed ML to ascertain the statistical effect on the model's suitability. The χ2/df ratio can be used as well, but it needs three values or less so that the model could be considered as acceptable. The χ2/df ratio requires smaller percentage values for better fit, for instance, the ratio of 2-5 (James et al., 1982). Meanwhile, the values of AGFI, GFI, NFI, IFI, TLI, and CFI were all between 0.80 and 0.90, and so, they were all acceptable. As for the attained RMSEA value to determine the goodness-of-fit of the model, it should fall within the range of 0.05-0.08 to be acceptable.   Table 5 provides the results in detail. Table 5 presents the outcomes of hypothesis testing (path coefficients-β), and all were supported. In general, the results showed that e-learning quality, contactless quality, e-learning cognitive involvement, and learning satisfaction had significant impacts on e-learning achievement of e-Learning students. The significant effect of elearning system quality on e-learning cognitive involvement and e-learning satisfaction was demonstrated by the results. Additionally, the results showed a positive impact of contactless learning quality on e-learning cognitive involvement and elearning satisfaction. As also shown by the results, e-learning cognitive involvement significantly and positively affected elearning satisfaction of students. Further, both e-learning cognitive involvement and e-learning satisfaction affected e-learning achievement of students.

Discussion and conclusion
The factors of e-Learning cognitive involvement and e-Learning satisfaction were added to e-Learning success factors, based on the IS success model. Relevantly, Huang and Chang (2004) reported learning involvement and satisfaction as crucial in educational research. Additionally, several of the findings in this study are considered as significant and are discussed in this section. First, this study found significant relationships. Specifically, e-Learning system quality (H1b) and contactless learning quality (H5b) were found to be significant factors in increasing satisfaction of students. In addition, e-Learning cognitive involvement (H2) was found to solidify e-Learning satisfaction. Furthermore, the results showed a positive and significant impact of e-Learning cognitive involvement and e-Learning satisfaction on e-Learning achievement (H3, H4). In other words, for students of e-Learning, increased involvement and satisfaction towards e-Learning leads to the achievement of educational goals, and the increased success of e-Learning (Seta et al., 2018). Additionally, outcomes of analysis showed significant difference in e-Learning achievement of the main e-Learning platform. Furthermore e-Learning system quality (H1a) positively affects e-Learning cognitive involvement. Second, the results showed the direct impact of contactless learning quality on e-Learning cognitive involvement, and so, H5a was supported. This finding could be linked to context-specific issues of e-Learning. e-Learning cognitive involvement was linked to Use in the IS success model and the results showed that e-Learning plays an adequate role in Use. Also, the finding implies the potential of other fitting variables like attendance and participation, that correspond to Use in the IS success model in e-Learning context. Third, the results demonstrated how COVID-19 pandemic has altered the e-Learning experience of students, particularly relating to contactless learning quality and e-Learning system quality. What can be construed is that the new normal has increased the success of e-Learning.

Implications for research
A number of implications of this study are discussed in this section. Firstly, the e-Learning success model grounded upon student experiences and the IS success model (DeLone & McLean, 2003) proposed in this study adds to the development of a comprehensive framework of e-Learning success in the new normal, in response to the recommendations made by Belleflamme and Jacqmin (2016). In increasing e-Learning understanding, this study obtained e-Learning experiential factors from past relevant studies and reviews. This study thus facilitates the real-life application of e-Learning success models in the education domain in the new normal. Secondly, this study has expanded the IS success model into the e-Learning context through the inclusion of the lecture construct, to bridge the gap mentioned by Wang et al. (2007). Furthermore, the e-Learning model proposed in this study is comprehensive as it takes into account all components of e-Learning (i.e., students, teachers and systems). Notably, e-Learning is an educational service that is more associated with lectures and instructors than the e-Learning system itself, while past studies were looking at service as merely services provided by the system. On the other hand, this study considered other aspects namely the system, the lectures, and instructors, from the perspective of students. Hence, this study presents a fresh and richer e-Learning success model. Additionally, post-hoc analyses were also performed in this study to ascertain the mediation effect of e-Learning cognitive involvement and e-Learning satisfaction on e-Learning achievement.

Implications for practice
A number of practical implications are discussed in this section. First, for e-Learning service providers, they have to understand the importance of system quality. As indicated by DeLone and McLean (2003), system quality affects IS use. Hence, e-Learning providers should be concerned with quality because a system of low quality will decrease satisfaction and involvement. Hence, e-Learning system providers should provide high-quality websites, user interface and videos. Second, this study found seven major experiential factors affecting students (from the perspective of students), and so, it is important to increase the involvement of students in their e-Learning course and improve their knowledge acquisition and skills, by way of e-Learning. Additionally, successful e-Learning systems can be achieved through the following: increasing the perceived relevance of a course (e-Learning cognitive involvement) by the e-Learning system operators, increasing course favorability (e-Learning satisfaction) and increasing students' perceived attainment (e-Learning achievement).

Limitations and future research directions
A number of limitations of this study have been identified. The first limitation concerns the data collection which involves different subjects. For instance, during the second stage, the university students enrolling in online courses in Jordan were recruited, and so, the results may be influenced by the Jordanian educational system -Jordanian educational system comprises more compulsory courses than electives. As such, the results may not show the impact of contactless learning quality on e-Learning cognitive involvement. Additionally, the study data need to be reanalyzed with a generalizable subject. Also, considering that e-Learning is dynamic, it would be useful to use a longitudinal study approach, so that the dynamic changes occurring to the students' e-Learning journey could be examined and ascertained. As an example, the study would collect data on participation rate of e-Learning, grades at the end of an e-Learning course. This would allow the actual achievement of students to be measured. Additionally, any causal relationships between e-Learning success factors could be ascertained.