An extended DeLone and McLean’s model to determine the success factors of e-learning platform

E-learning

According to Choudhury & Pattnaik (2020), e-learning is defined as “the transfer of knowledge and skills, in a well-designed course content that has established accreditations, through an electronic media like the Internet, Web 4.0, intranets and extranets”. E-learning is an educational system that provides training and education and it is web-based to disseminate knowledge, information and communication between stakeholders (Cidral et al., 2018). E-learning is being adopted and widely used at all levels in education: universities, secondary schools and primary schools (Hubalovsky, Hubalovska & Musilek, 2019). E-learning enables learners to improve and develop by exploiting its characteristics in educational processes (Choudhury & Pattnaik, 2020). E-learning has advantages such as improving students’ study skills by engaging them ineffective study and allowing students to study anywhere and anytime (Wu et al., 2020). Al-Fraihat et al. (2020) reported that education institutes broadly adopt e-learning. For example, in the U.K., 95% of higher education institutes have already adopted and used learning management systems (L.M.S.) in their educational processes. Smart technology has already impacted teaching methods, learning and education. Consequently, e-learning is expanding quickly due to these technologies and devices’ access to learning resources (Al-Fraihat et al., 2020).

The authors in Gul et al. (2017) have proposed a mind map-based method for improving the students’ learning skills. They concluded that mind map-based methods worked well for both text and block-based languages to improve student’s learning outcomes.

Theoretical framework

In the past, the study of information systems success was inaccurate and scarce due to the complex nature of the systems (Petter, DeLone & McLean, 2008). In 1992, DeLone and McLean’s Information Systems Success (D&M ISS) model was proposed to determine the success of information systems. They focused on six factors: system quality, information quality; use; user satisfaction, individual impact, and organizational impact (Yakubu & Dasuki, 2018). DeLone and McLean revised the model in 2003 to study researchers’ feedback regarding the weaknesses and strengths of the model. In this phase, they grouped individual and organizational impact as net benefits and added a new factor, service quality, because service is essential in I.S. success. They also divided the use, intention to use and actual use (Yakubu & Dasuki, 2018). According to DeLone & McLean (2003), system quality refers to system performance, technical characteristics and user-friendliness. Service quality refers to the extent of technological staff response and competence. Information quality refers to how the system content is accurate, valid and available. Use is considered the first phase of system success because effective use means full adoption. Users’ satisfaction refers to users’ degree of satisfaction and acceptance of the system. Net benefits are defined as “the perceived individual and organizational impacts on tasks/job performance and efficiency” (Cidral et al., 2018).

This study’s research model was a modified and updated version of the D&M-IS success model (Fig. 1). In addition, three constructs were added: (1) perceived usefulness from the Technology Acceptance Model (T.A.M.); (2) Learner quality; and (3) Instructor quality. Those constructs were adopted in other studies of e-learning, such as those by Cidral et al. (2018), Al-Fraihat et al. (2020), and Ozkan & Koseler (2009).

Information quality

Information quality is considered an essential factor in assessing the e-learning system’s success. It plays a vital role in the learning process and may cause complex problems if poor information quality (Al-Sabawy, 2013). In the updated DeLone and McLean’s Information Systems Success (D&M ISS) model (DeLone & McLean, 2003), there is a relationship between information quality on the one hand and use and user satisfaction on the other. The study by Rai, Lang & Welker (2002) revealed that information quality was significantly related to use. Those results were consistent with the studies of Halawi, McCarthy & Aronson (2008) in knowledge management systems and Kositanurit, Ngwenyama & Osei-Bryson (2006) in health information systems. In addition, a significant relationship was found between information quality and perceived satisfaction and usefulness (Seddon & Kiew, 1994; Seddon, 1997).

In the e-learning context, some empirical studies have measured the relationship between information quality on the one hand and both use and perceived satisfaction and perceived usefulness on the other. For example, Klobas & McGill (2010) and Eom et al. (2012) found information quality was significantly related to use and satisfaction in L.M.S. Lwoga (2014) found a significant relationship between information quality and perceived usefulness in web-based L.M.S.s. This was consistent with the findings of Chen (2010) in e-learning systems. A recent study by Al-Fraihat et al. (2020) found that information quality positively impacted perceived satisfaction and usefulness of the e-learning system in a U.K. university. Consequently, the following is hypothesized:

• H1: Information quality will have a significant positive impact on the perceived usefulness of the Shaqra University e-learning platform.

• H2: Information quality will have a significant positive impact on using the Shaqra University e-learning platform.

• H3: Information quality will significantly impact the perceived satisfaction with the Shaqra University e-learning platform.

Instructor quality

Instructor quality has a crucial role in successful e-learning and it has attracted the interest of many researchers in this context (Al-Fraihat et al., 2020). Sun et al. (2008) found a significant positive relationship between instructor response timeliness, instructor attitude toward e-learning, and satisfaction with e-learning at two public universities in Taiwan. Cidral et al. (2018) found a positive relationship between instructor attitudes toward e-learning and satisfaction with e-learning in the Brazilian context. Lwoga (2014) found that instructor quality was significantly related to perceived usefulness and user satisfaction with L.M.S. in Tanzania. Mtebe & Raphael (2018) found that instructor quality significantly impacted learners’ satisfaction with the e-learning system at the University of Dar es Salaam, Tanzania. Park (2009) and Roca & Gagné (2008) found that subjective norms, considered an indicator of instructor quality, were significantly related to usefulness and satisfaction. McGill & Klobas (2009) found a significant positive relationship between instructor norms and L.M.S. utilization. A recent study by Al-Fraihat et al. (2020) found that instructor quality positively impacted perceived usefulness and satisfaction of the e-learning system in a U.K. university. Consequently, the following are hypothesized:

• H4: Instructor quality will have a significant positive impact on the perceived usefulness of the Shaqra University e-learning platform.

• H5: Instructor quality will have a significant positive impact on using the Shaqra University e-learning platform.

• H6: Instructor quality will significantly impact the perceived satisfaction with the Shaqra University e-learning platform.

Learner quality

According to Al-Fraihat et al. (2020), this construct has been successfully adopted in many studies of e-learning. Some previous research studied the subset of the learner quality construct. Park (2009) and Ong, Lai & Wang (2004) found that the learner’s self-efficacy was significantly related to perceived usefulness. McGill & Klobas (2009) found a significant relationship between learner attitude toward L.M.S. and L.M.S. utilization. In addition, it has been found that a significant relationship existed between student involvement and both use and satisfaction of L.M.S. in an Australian university (Klobas & McGill, 2010). A recent study (Castiblanco Jimenez et al., 2021) found that learners’ computer anxiety significantly influenced perceived usefulness in an e-learning tool designed for E.U. farmers and agricultural entrepreneurs. In studies by Ozkan & Koseler (2009) and Sun et al. (2008), there was a significant relationship between learner quality and perceived satisfaction. A recent study by Al-Fraihat et al. (2020) found that learner quality positively impacted the use and perceived usefulness and satisfaction of the e-learning system in a U.K. university. Consequently, the following are hypothesized:

• H7: Learner quality will have a significant positive impact on the perceived usefulness of the Shaqra University e-learning platform.

• H8: Learner quality will have a significant positive impact on using the Shaqra University e-learning platform.

• H9: Learner quality will significantly impact perceived satisfaction with the Shaqra University e-learning platform.

System quality

There are relationships between system quality, use, and user satisfaction in the updated DeLone and McLean Information Systems Success (D&M ISS) model (DeLone & McLean, 2003). Some previous studies found a significant positive relationship between system quality and use in the information system context (Halawi, McCarthy & Aronson, 2008; Hsieh & Wang, 2007; Iivari, 2005). It was also found that system quality was significantly related to use in an e-learning context (Balaban, Mu & Divjak, 2013; Garcia-Smith & Effken, 2013; Lin, 2007; Marjanovic, Delić & Lalic, 2016).

Some researchers in Gul et al. (2017) worked on the role of IoT in the education system for improving the quality of education in higher educational institutions. According to them, the paradigm shift of IoT will bring new opportunities for e-learning systems.

Some earlier studies found a positive relationship between system quality and user satisfaction (Halawi, McCarthy & Aronson, 2008; Hsieh & Wang, 2007). It was claimed that users would be satisfied with the e-learning system if it had a technical quality (Hassanzadeh, Kanaani & Elahi, 2012).

Studies by Seddon & Kiew (1994) and Seddon (1997) found that system usefulness will increase if the system quality increases. They also found that system quality is a crucial predictor of usefulness. This is consistent with previous studies by Sabherwal, Jeyaraj & Chowa (2006) and Liaw (2008). A recent study by Al-Fraihat et al. (2020) found that Technical System Quality positively impacted the perceived satisfaction and usefulness of the e-learning system in a U.K. university. Consequently, the following are hypothesized:

• H10: System quality will have a significant positive impact on the perceived usefulness of the Shaqra University e-learning platform.

• H11: System quality will have a significant positive impact on using the Shaqra University e-learning platform.

• H12: System quality will significantly impact the perceived satisfaction with the Shaqra University e-learning platform.

Service quality

The construct of service quality is essential in the DeLone & McLean model (2003). They assumed relationships between service quality and use and user satisfaction. This construct was used in many information system studies such as Chen & Cheng (2009) who found a relationship between service quality and satisfaction in an online shopping system. It was also found that a significant relationship existed between service quality and use in an e-government system (Wang & Liao, 2008). The results of a study by Ozkan & Koseler (2009) and Roca & Gagné (2008) revealed a significant relationship between service quality and satisfaction in an e-learning context. Previous studies by Al-Sabawy (2013) and Ngai, Poon & Chan (2007) found a significant relationship between perceived usefulness and service quality. A recent study by Al-Fraihat et al. (2020) found that service quality positively impacted perceived satisfaction with the e-learning system in a U.K. university. Consequently, the following are hypothesized:

• H13: Service quality will have a significant positive impact on the perceived usefulness of the Shaqra University e-learning platform.

• H14: Service quality will have a significant positive impact on using the Shaqra University e-learning platform.

• H15: Service quality will significantly impact the perceived satisfaction with the Shaqra University e-learning platform.

Perceived usefulness PU

Davis (1989) used usefulness as a critical construct in the technology acceptance model (T.A.M.). Davis (1989) defined perceived usefulness as “the degree to which a person believes that using a particular system would enhance his or her job performance.” Previous studies (Roca & Gagné, 2008; Davis, 1989) claimed that acceptance is critical in measuring e-learning systems’ success. For example, the study by Arbaugh (2000) found a positive relationship between the perceived usefulness of the course software and student satisfaction with an Internet-based course. Other studies revealed that perceived usefulness significantly impacted user satisfaction (Al-Sabawy, 2013; Seddon, 1997; Limayem & Cheung, 2008). According to Al-Fraihat et al. (2020), students used the e-learning system if they perceived it was helpful. In the e-learning context, some studies measured the relationship between perceived usefulness and the use of e-learning systems (Islam, 2013; Pituch & Lee, 2006; van Raaij & Schepers, 2008; Sandjojo & Wahyuningrum, 2015; Šumak, Heričko & Pušnik, 2011).

The study by Hwang et al. (2008) found that a relationship between usefulness and net benefits in a Taiwanese hospital and the study by Park et al. (2011) found a significant relationship between usefulness and organizational benefit. Gul et al. (2017) found a significant relationship between usefulness and personal impact. The study results by Hasan, Gholamhosseini & Sarkar (2017) revealed a significant relationship between usefulness and (individual and organizational impact). A recent study by Al-Fraihat et al. (2020) found that P.U. positively impacted perceived satisfaction, benefits, and use of the e-learning system. Consequently, the following are hypothesized:

• H16: perceived usefulness will have a significant positive impact on the benefits of the Shaqra University e-learning platform.

• H17: perceived usefulness will have a significant positive impact on using the Shaqra University e-learning platform.

• H18: perceived usefulness will significantly impact perceived satisfaction with Shaqra University’s e-learning platform.

Perceived satisfaction

Perceived satisfaction was considered an essential factor in measuring information system success. Based on the DeLone and McLean information systems success model (2003), perceived satisfaction significantly impacted benefits (Al-Fraihat et al., 2020). Hassanzadeh, Kanaani & Elahi (2012) asserted that the system will be used and benefits will be gained if users are satisfied when using the system. Cidral et al. (2018) and Hassanzadeh, Kanaani & Elahi (2012) found a significant relationship between perceived satisfaction and individual impacts. A recent study by Al-Fraihat et al. (2020) found that perceived satisfaction toward the e-learning system had positive benefits for students in a U.K. university. Consequently, the following is hypothesized:

• H19: Perceived Satisfaction with the Shaqra University e-learning platform will have a significant positive impact on benefits for students.

Actual use

The Actual Use construct measures information system success in the DeLone & McLean (2003) model and measures acceptance in the T.A.M. model (Davis, 1989). Previous studies have found that the use of the system was significantly related to its benefits (Garcia-Smith & Effken, 2013; Chen & Tseng, 2012; Hou, 2012). A moderate relationship between using the system and its benefits was found by Petter, DeLone & McLean (2008). The use of e-learning systems in training courses positively influenced net benefits for company employees (Chen & Tseng, 2012). Those results were consistent with those in studies by Halawi, McCarthy & Aronson (2008) and Kositanurit, Ngwenyama & Osei-Bryson (2006). A recent study by Al-Fraihat et al. (2020) found that use has a positive impact on the benefits of the e-learning system in a U.K. university. Consequently, the following is hypothesized:

• H20: Actual use of the Shaqra University e-learning platform significantly impacts the system’s benefits.

Demographic data

The results in Table 1 indicate that most participants (53.7%, n = 314) were female while 46.3% were male (n = 271). In addition, the majority of participants were studying for a Bachelors’s degree (n = 527, 90.1%), while the remainder (n = 38, 6.5%) and (n = 20, 3.4%) were studying for diplomas and Masters degrees, respectively.

Testing the goodness of the measurement model (outer model)

This section describes the instrument testing steps. PLS-SEM was used as a primary methodology to evaluate the model used in the study due to its sophistication—nine structures, 34 indicators, and 20 interactions—and PLS-SEM suits those models (Hair et al., 2010). The calculation and structure of the model were tested using SmartPLS version 3.3.3. As per Wong (2013), it is necessary to adequately assess validity and reliability before testing the path coefficients in the structural model. Therefore, we used a criterion adopted by Wong (2013) (see Table 2).

Reliability

According to Wong (2013) and Sekaran (2003), reliability measures the consistency and stability of the instrument. Thus, as shown in Fig. 2, internal consistency and indicator reliability are required.

Indicator reliability

The loadings of indicators with respective latent variables are used to assess the reliability of an individual item. To examine the reliability of any individual item, loading and indicator correlations with respective latent variables need to be checked (Wong, 2013). An item’s loadings must be greater than 0.70 to achieve a satisfactory result (Hair et al., 2010). This means that the inclusion of poor indicators may lead to poor fitting in the covariance-based S.E.M. analysis. Accordingly, all items’ loadings were checked, and items with a loading less than 0.70 were deleted.

Internal consistency reliability

To assess the internal consistency reliability of variables, the composite reliability needs to be checked and the variables’ composite reliability must be at least 0.7 (Wong, 2013). Some researchers argue that Cronbach’s alpha must be for variables that need to be checked in addition to the composite reliability of variables to properly examine the Internal Consistency Reliability of variables, with both required to be higher than 0.7 (Hair et al., 2012; Fornell & Larcker, 1981). In Table 3, both composite reliability and Cronbach’s alpha are at acceptable levels.

Validity of scales

To ensure that the model’s reflective measurements were adequate, its discriminant and convergent validity were tested. In addition, the unidimensionality and relevance of the scales were analyzed and the corresponding coefficients were tested. Furthermore, corroborative factor analyses were used to evaluate the merged and discriminating viability of the estimate scales. Finally, the constructions’ validity was evaluated by confirmatory factor analysis (C.F.A.). All of these are discussed below.

Convergent validity

Convergent validity is used to examine a model’s capacity to explain the indicator’s variance, and the average variance extracted (AVE) value is used to provide evidence for convergent validity (Wong, 2013; Fornell & Larcker, 1981). The AVE value must be at least 0.5 to ensure that convergent validity is at the acceptable level (Wong, 2013). Table 3 shows that all variables’ AVE values are higher than 0.5, which means all variables’ convergent validity is confirmed.

Discriminant validity

The degree of distinction of objects between structures is discriminant validity (Hair et al., 2014). Furthermore, discriminant validity can be regarded as a statistical metric for examining the level of differentiation between items and constructs. According to Hair et al. (2010), it is better to consider a phenomenon rather than other buildings due to the highly discriminant validity of a building. In this analysis, the validity of the discriminant measurements was checked by the overlap in the variance to ensure the correlation between measurements for each category and the building itself and to make sure that there were no problems in the overload for the measured objects. The discriminant validity of our analysis was measured by two separate metrics, namely, Fornell-Larcker and cross-loading criteria. The Fornell-Larcker criterion is a normative and conservative approach to assessing discrimination (Hair et al., 2016). For this criterion, the independence of a latent variable (equivalent to the square root of the AVE) must be greater than its correspondence with all other latent variables to maintain a discriminant validity. As seen in Table 4 below, the values of the principal diagonal (the self-correlations) are higher than any other value in the same range and column and all latent variables in the model follow this criterion.

The cross-loading analysis is the second tool used to assess discriminant validity. Chin (2010) states that the loading value can surpass their loading with other latent variables for each object with its associated latent variable. All elements meet this test, as seen in the cross-loading matrix in Table 5.

Testing the structural model (inner model)

The effect size (f2)

The effects of R2 variables were calculated by evaluating the effect size (f2) after the assessment of R2 to ascertain whether the effects on the endogenous variable of a given exogenous variable are significant. As recommended by Hair et al. (2014), the following formula was used to calculate f2:

(1) $$f^2=\frac{R_{included}^2-R_{excluded}^2}{1-R_{included}^2}$$

In both cases, R2 included reflects the condition in which the exogenous latent predictor is a component of the structural model. At the same time, R2 omitted corresponds to values in which the latent exogenous predictor is deleted from the structural model. Ahmad et al. (2020) used the operational concept of multiple regression as a guideline for determining if an exogenous preview component has a big, low, limited, or no impact (f2). This description defines all values below 0.02 as having no effect, values between 0.02 and 0.15 as having a small effect, values between 0.15 and 0.35 as having medium effects and values above 0.35 have a significant impact scale. Table 7 shows the f2 values for this analysis.

Predictive relevance of the model (Q2)

Predictive validity is another criterion for the consistency evaluation of the structural model (Chin, 2010). Predictive significance is founded on the premise that all indices of an endogenous latent variable must be predictable by the model (Maqsood et al., 2020). The blindfolding protocol was performed to determine Q2 to calculate the commonality of cross-validity (cv-comm) and redundancy of cross-validity (cv-red).

A fixed distance, D, is used to extract data from the data collection through blindfolding procedures. The value of D can be between 5 and 10 (Chin, 2010). The sample size, n, divided by D must be a complete number. The model parameters are then calculated to expect those data volumes to be removed and treated as missing values. However, blindfolding happens only in the case of an endogenous latent variable with reflective dimensions, as in this analysis (Maqsood et al., 2020). Maqsood et al. (2020) supports cross-validity redundancy as it forecasts both the metrics model and the data projection systemic model, which is PLS-SEM-compatible. Bagozzi (1994) argued that the redundancy value of cross-validity above zero (Q2 > 0) would be predictive. On the other hand, a Q2 value below zero shows that the model is not predictable. As seen in Table 8, all endogenous latent variables were greater than negative in the cross-validity redundancies (Q2). This research model therefore has an adequate predictive potential.

The goodness of fit of the model (GoF)

The goodness of fit (GoF) is a global fitness measure (Tenenhaus et al., 2005). The geometric mean indicates that the endogenous variables have an average of R2 and AVE. Its purpose is to analyze the design and calculation of the model, while the actual output is the subject of the model (Chin, 2010; Henseler & Sarstedt, 2013). The calculation method for GoF is as recommended by Hair et al. (2014):

(2) $$GOF=\sqrt{\overline{(R^2}\ast \overline{AVE})}$$

(3) $$GOF=\sqrt{\left(0.838\ast 0.864\right)}=0.851$$

Wetzels, Odekerken-Schröder & Van Oppen (2009) provide the GoF criteria for determining if its values are regarded as small, medium, or large, where values under 0.10 can be considered as small, and values between 0.10 and 0.25 can be considered as medium and values to 0.35 can be considered as significant. In this study, GoF was 0.586, meaning it was sufficiently large enough to ensure the validity of the P.L.S. model.

Structural model analysis

The relationships between contingent and autonomous buildings derived from calculation models (C.F.A. models) can be explored by scientists on a structural basis. Even so, for the testing of relations between the structures before analysis, it is essential to establish a relationship or a hypothetical model. After real-world findings and a literature review, the hypothetical model was proposed. A path analysis was conducted on the hypothetical model to approximate the coefficients and importance of the relations. Finally, the fit indices for the model were compared to standard fit indices to validate the model’s suitability.

Collinearity assessment

The systemic model must be assessed before a verdict is reached. In the structural model, collinearity may be a challenge. In more detail, a value of 5 and a more excellent Variance inflation factor (V.I.F.) means that a problem may arise (Maqsood et al., 2020). The findings of the collinearity evaluation are summarized in Table 9. As all V.I.F. values are less than 5, there is no evidence of collinearity between predictor sets.

Structural model result

The findings of the proposed research model will be discussed in this section via the modeling of structural equations (S.E.M.). The literature review shows that using S.E.M.s is common in behavioral science research, especially in IT/IS (Gefen, Straub & Boudreau, 2000). The Smart P.L.S. package using Partial Least Square (P.L.S.) method has applied S.E.M. Fornell, Lorange & Roos (1990) and Wold (1985) argue that P.L.S. is the ideal approach for the exploration of complex models of latent variables.

Hypothesized structural model

Inner model path coefficient sizes and significance

The structural model, which is well suited to the results, has been studied as the basis for accepting or rejecting the hypothesized relationship, standardized route coefficients and p-values. The results shown in Fig. 2 and the theory related to individual model paths are summarized in Table 10. The 5% reliable standard supports assumptions with a value of p of the corresponding direction above 0.05 (Hair et al., 2016).