Models for Measuring E-Learning Success in Universities: A Literature Review

It is obvious that in the Internet era the higher education institutions (HEIs) must innovate the services they offer by integrating ICT (Information and Communication Technology) in the learning process. According to the theoreticians and practitioners insights in the matter, the e-learning systems offer many advantages and compensate the weaknesses of the traditional learning methods. In consequence, it emerged the need for developing a model that measures the success of the e-learning systems. This paper presents results of the research conducted in order to develop a comprehensive model for measuring e-learning system success in universities.


Introduction
The fundamental terms that define the paradigm of the contemporary society are information, knowledge and communication.The European Programs (FP6, FP7) are issued in order to sustain, between 2010 and 2030, the passage from an information society to a knowledge society.The formation and consolidation of a knowledge society implies setting priorities.Among these priorities, I mention the investment in education, which aims at increasing the level of human capital, the fluidization of the knowledge use and dissemination, as well as the existence of an efficient innovative system that should include universities that allow the increase of the knowledge global stock, the assimilation, adjustment, creation of new technologies and the development through the use of ICT.The knowledge society provides new dimensions to the learning process and, as such, in the higher education system it emerges the need to introduce modern teaching techniques, based on the use of the ICT.In order to achieve the educational reform aligned with the European standards, to withstand the challenges of an increasingly competitive environment and to increase the academic reputation, the Romanian universities perform steps for a strategic shift by acknowledging the university-knowledge binomial and thus innovate the services they offer.The students' learning experience continues to be influenced by the modality in which teachers integrate information and ICT in their university courses.Since it is obvious that the use of the ICT is not absolutely necessary only for universities that provide distance learning programs and it is more and more encouraged in order to improve students' experience, universities have focused their attention to determine a model for integrating ICT in the learning process, so as to provide students know-how and skills adapted to our current and future society.Thus, we notice that universities make efforts to implement or develop elearning systems adapted to their organizational structure and to use blended learning in their academic programs.The e-learning systems offer significant improvements to the learning process and considerably reduce the negative effects of the singular application of traditional teaching methods.Hence, the success of e-learning systems implementation (and its measurement) is imperative:  to be able to determine their added value;  to understand the overall effect on the activity and the learning process within the HEIs, and  to justify the investment into such systems.

E-learning, E-Learning Systems and Blended Learning
In a broad sense, by e-learning, one understands the totality of education situations where the ICT is significantly used.In a restricted sense, e-learning is a type of distance education, as a planned teaching-learning experience organized by a HEI that provides (using an e-learning technology and the web browser as the main interaction tool) study materials in a sequential and logical order, so as to be assimilated by students in their own manner.
The e-learning technology refers to an online system that replicates and adapts the traditional didactic endeavor components: planning, specific content and methodology, interaction, support and assessment.The elearning systems are platforms facilitating the learning process [3] that aim at the learning process flexibility [4] and the adaptation of the teaching methods to the individual learning style of students.Because of the significant differences between traditional teaching methods and online teaching methods, the conversion of the traditional courses into online courses needs careful planning, monitoring and control [5].
In the academic literature, there is only one globally valid definition that can comprise all the aspects related to the blended learning concept.Blended learning (or combined/hybrid/integrative) aims at achieving the learning objectives through the application of specific technologies in order to customize the act of learning and to transfer knowledge and skills to the right person at the right time."Blended" means the combination of several teaching methods: asynchronous and synchronous, off-site and onsite, offline and online, individual and collaborative, structured and non-structured.The biggest advantage of applying the blended learning concept is the adapting of teaching methods to the learning individual style.An individual learning style may be defined as the user's (student's) preferred modality (by means of study or experience) to assimilate new knowledge and skills.This does not mean that all the knowledge and know-how of the student will be assimilated in the same manner, but only that to him/her this learning style is more efficient [1].By adapting the learning process to the students' individual learning styles, two important benefits are obtained: students become aware of their individual learning style (an important factor of personal and professional development) and the improvement of the student's response to the learning process [2].

Measuring the Success of the E-Learning Systems
Assessing the success of e-learning systems was done by using multiple criteria and approaches especially due to the numerous ways of defining the e-learning term [10], [55].I have identified four such approaches in the academic literature:  the DeLone and McLean (D&M) model;  the TAM model (Technology Acceptance Model);  models focused on users' satisfaction;  models focused on the e-learning quality.

The D&M Model
The success and quality of an e-learning system may be measured in the same manner like the success and quality of an IS (Information System).One of the most prolific models that have been used for measuring the success of an elearning system is the D&M model, which was first presented in 1992 and, since then, it has been used in over 300 scientific papers [6].This model includes six components: the quality of the system, the quality of the information, the use of the system, the user satisfaction, the individual impact and the organizational impact; the relations between them are emphasized in Figure 1.For the time being, the D&M model includes six dimensions [6], [7], [8]: the system quality; the information quality; the services quality; the use of the system/the intention to use the system; the user satisfaction; the benefits of using the system.The relations between the model components are presented in Figure 3.The brief description of each component is to be found in Table 1.[21].

Component Description The system quality
The system performance.

The information quality
The quality of the system output.

The services quality
The efficiency of the support services provided for the system users.The intent to use| The use of the system The perceived behavior of system use| The actual behavior of system use.

The user satisfaction
The general perspective of users on the system.Benefits The advantages of using an IS.
 The quality of the education services provided by the system, with the following indicators standing out: the existence of some functionalities like chat, forum, etc. into the system [31]; communication facilities among students [45], [31], [46]; the adaptability of the system to individual learning styles [47], [48]; facilities for active and collaborative learning [49];  The attaining of goals, with the following indicators standing out: learning objectives accomplishment [50]; personal objectives accomplishment [50], [51].

The TAM model
TAM is an IS (Information Systems) theory that models how users accept and use a new technology.The first release of this model was created by Fred Davis in 1989 [56].This model (Figure 7) suggests that the users are influenced by certain factors when they de-cide upon how and when they are going to use the new technology:  Perceived usefulness or the extent to which a user believes that, by using a certain system, he/she will obtain increased professional performance;  Perceived ease-of-use or the extent to which a user believes that he/she will make considerably less efforts, by using this system, in order to fulfill his/her current tasks.
In the academic literature, between 1985 and 2013, there were published over 10 scientific papers that aimed at developing the model; over 20 scientific papers published extensions of this model, and over 30 scientific papers emphasizing its applicability (an excerpt from these papers may be found in [9]).
The second version of the TAM model was issued by Venkatesh and Davis, and published in 2000 [9] (Figure 8).[16] adopted the TAM model and made some essential changes on it, with a view to increasing its relevance in the context of the assessment of the e-learning systems (like the creation of a four-dimension model: trainer, student, technology and support, and their separate analysis [42] or the introduction and analysis of the interdependences of some components like "Attitude" and "Technical support" [16]).DOI: 10.12948/issn14531305/18.3.2014.07Fig. 9.The Roca model (adapted after [23]).
In 2008, Venkatesh and Bala published version 3 of the TAM model adapted for ecommerce, which includes the effects of components like "Trust" and "Risk" on the system use degree [44].

Models focused on the users' satisfaction
The user's satisfaction is considered to be one of the most important criteria that may be applied for the assessment of an e-learning system's success.In 2008, Sun et al. [26] classified the critical factors leading to the success of an e-learning system, using six dimensions: student, trainer, course, technology, design and context (Figure 10).Fig. 10.Dimensions of perceived e-learner satisfaction [26] Other studies that used this approach for measuring the success of an online training system were performed by Shee and Wang [12] in 2008 or Wu et al. [27]

Models focused on the e-learning quality
Scientific papers that adopted this approach for measuring the success of an e-learning system focused on the overall quality of the system and not only on the quality of the services.Such a model was created by Lee and Lee in 2008 [4] (Figure 12).

Conclusions
In this paper I presented previous studies conducted in order to provide models for evaluating the e-learning systems success.Even if these identified models were updated several times over the years, by many researchers, there is still room for improve-ments and extensions [9], [52].In consequence, by combining these previous models, I propose a more comprehensive model -ELSS.The testing of the model's hypothesis and the overall validation will be the subject of a future study.This paper was co-financed from the European Social Fund, through the Sectorial Operational Programme Human Resources Development 2007-2013, project number POSDRU/159/1.5/S/138907"Excellence in scientific interdisciplinary research, doctoral and postdoctoral, in the economic, social and medical fields -EXCELIS", coordinator The Bucharest University of Economic Studies.