Organizational Approach to the Ergonomic Examination of E-Learning Modules

. With a signiﬁcant increase in the number of e-learning resources the issue of quality is of current importance. An analysis of existing scientiﬁc and methodological literature shows the variety of approaches, methods and tools to evaluate e-learning materials. This paper proposes an approach based on the procedure for estimating parameters of local factors and receiving the integral index of usability quality of e-learning modules. We present a mathematical model which serves as a basis for the automated procedures for expertise. The use of fuzzy logic allows to reduce greatly the complexity of evaluating the formation of a repository of e-learning modules. The proposed approach is focused on the situation, when the university − has amassed a large number of e-learning modules that have to be assessed in terms of ergonomics; − is able to use experts in ergonomics and organization of e-learning (the experts can provide, as a rule, qualitative assessment); − is limited in resources on the development of special software for evaluation of e-learning modules; − is forced by the need to reduce the cost of expertise to be limited to considering only the main quality indicators that have the greatest impact on the ergonomics of e-learning modules. For automation of the ergonomic examination procedures a MatLab system is used, in particular Fuzzy Logic Toolbox.


Introduction
The use of information technology is an integral part of modern education.
In terms of active use of information technology in education e-learning resources are dynamically developing.A variety of commercial companies (e.g., Microsoft, Harbinger Knowledge Products, Algoryx, Technomatix) are involved in the development of elearning.This process also involved higher educational institutions (for example, Massachusetts Institute of Technology, Monterey Institute for Technology and Education, The Open University and The Russian National Open University "INTUIT").
In Ukraine, Sumy State University (SSU) is one of the leaders in the development of elearning modules of the unified educational e-learning system of the university (Vasylyev et al., 2007;Lavryk et al., 2007).Among the SSU innovations there are various types of e-learning modules: an electronic synopsis, a test, a computer tutor, a computer learning game, a virtual lab (http://fpkpi.sumdu.edu.ua/en/nav-main-en.html).
The developed e-learning modules are related to the following areas: humanitarian, natural science, mathematics, economic, medical, and engineering.The link http://fpkpi.sumdu.edu.ua/en/int_tool.htmlrefers to the some elearning modules examples such as the virtual lab "Analog hardware complex for simulating dynamical systems on their structural models", biochemistry laboratory-based work "Method of Iyendrashek", virtual operation "Aortocoronary Bypass", English language learning game.
At present there are widely spread open educational resources which can be accessed via special repositories.Examples of such repositories are international, national and institutional repositories like: MERLOT, EducaNext, ARIADNE, EduTube, The National Repository of Online Courses, Connexions, The Russian Federal Center of Information and Educational Resources.
A significant increase in the number of developed commercial, public or local resources of e-learning poses a difficult choice to users.One of the decisive factors influencing the choice may be the quality of the resource.For example there might be the repositories of the authoritative institutions of higher education.

Problem Formulation
In the practice the e-learning laboratory of Sumy State University was faced with such a situation: − on the one hand (estimates vary), one of the largest banks of e-learning modules in Ukraine is accumulated at the University; − on the other hand, e-learning modules were developed at different times by different teachers, without uniform standards and have a different quality.The analysis shows that there are alternatives to e-learning modules and the extent of their use by students, who have the ability to choose, is very different.Moreover, this difference is not so much the content, but the ergonomics.
The acuteness of the situation was aggravated by: − a large number of foreign students (more than 1000); − implementation of a number of international projects for open educational electronic resources.Many of the modules to be simply removed from the bank of the electronic resources.However, this must be done, on the one hand, by the system of the formal parameters, and on the other hand -the backing-out procedure must have a powerful "explanatory function" -type of "what modern ergonomic requirements for qualitative e-learning modules are".
Thus, the problem can be described as follows.It is necessary: − to exclude from the learning process the e-learning modules of poor quality in the view of ergonomics; − to include in the technology of the development of new e-learning modules a mandatory step of acceptance, which in its part, includes an ergonomic expertise.Procedure requirements of the ergonomic expertise: − automation of conducting an ergonomic assessment; − compliance with existing technologies and the international practice; − the ability to communicate with experts in a language approaching natural; − the possibility of an integrated assessment of ergonomic quality based on expert evaluation of local indicators; − the maximum reduction in the complexity of the examination without significant loss of its quality; − exclusion from consideration groups of parameters, which are of the least significant impact on the overall ergonomic evaluation; − elimination of the need to purchase (design) special software for examination; − the use of widely recognized and available means of automation; − the availability of explanation (why making this or that rating).
Thus, the task is to develop an ergonomic expertise process that meets put requirements.

Usability Evaluation Methods
Usability as defined in ISO 9241-11:1998 is understood as the "extend to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use".The term "Ergonomics" has a wider meaning and has been introduced in the ISO 6385 in the year 2004: "Scientific discipline concerned with the understanding of interactions among human and other elements of a system, and the profession that applies theory, principles, data and methods to design in order to optimize human well-being and overall system performance".
A large set of evaluation techniques exist in the literature.Preece et al. (2007) clustered them in three main approaches: usability testing, analytical evaluation and field studies.Usability testing concerns the analysis of users' performance on the tasks for which the system is designed (Preece et al., 2007).This approach has the potential to provide reliable results since it involves samples of real users.However, reproducing realistic situations of usage in a laboratory is difficult, e.g., selecting a representative sample of users and tasks, training users to master advanced features of the system in a limited time period, or weighting the effect of important contextual factors on their performance (Lim et al., 1996).
Analytical methods are done by usability experts, who put themselves in the intended end-users position.Based on the experts expertise and usability heuristics the expert validates the software (Blecken et al., 2010), and as no user needs to be involved, these evaluation methods fits best early in the development process.Examples of analytical methods are "Guidelines", "GOMS" or "Heuristic Evaluation" (Bernérus A. et al., 2010).The analytical approach includes inspections and the application of formal models to predict users' performance.A common inspection method is heuristic evaluation (Nielsen, 1993).It involves experts who inspect the system and evaluate the interface against a list of usability principles, i.e., the heuristics.The main advantage is related to cost-saving: they "save users" and do not require special equipment or lab facilities (Jeffries and Desurvire, 1992).In addition, experts can detect a wide range of problems of complex systems in a limited amount of time.The main drawback of such a technique is the dependency on the inspectors' skills and experience, as heuristics are often generic and underspecified (Doubleday et al., 1997;Law, 2007).
Field studies differ from the other evaluation approaches because they are conducted in natural settings.Their aim is to understand what users do naturally and how technology impacts on them (Lanzilotti et al., 2010).They are useful for identifying opportunities for new technology, eliciting requirements, deciding how best to introduce new technology, and evaluating technology in use.Evaluating usability in the field is difficult, due to the complexity of the environment and the activities to be observed, and to the large amount of data to be analysed (Pascoe et al., 2000).
With respect to evaluating the quality of open educational resources in practice there are used the method of content reviewing prior to its publication (the experience of MIT Open CourseWare) or the model of "lenses" for peer evaluation and quality control after the open publication (Connexions experience) (Baraniuk, 2008, pp. 232-236).
An analysis of existing scientific and methodological literature shows the variety of approaches, methods and tools for evaluation of e-learning materials.There exist several models of evaluation from different perspectives (pedagogical and technical), for example Kirkpatrick's Four Levels of Evaluation (Galloway, 2005), Analytic Hierarchy Process approach by Colace et al. (2006), and also Pedagogy Effectiveness Index (Sonwalkar, 2002).Oliver et al. (2000Oliver et al. ( , 2007) ) provide a comprehensive account of the development of the field of evaluation of e-learning during the last decade, describing checklists, guidelines and toolkits designed to facilitate evaluative work of the academic staff.
The large number of literature on the subject of quality assessment shows interest and the need for more affordable and practical means of evaluation.For example, in the materials of the report "Evaluation of e-learning courses" (Jara et al., 2008, p.12) the emphasis is placed on the analysis of the existing evaluation tools specifically for a teacher, who is often not an expert in the field of quality assessment.The most common method of evaluating the quality of e-learning modules is a method of peer review.Experts are offered a variety of questionnaires, checklists in electronic or paper form.
Despite several studies have been performed to compare different approaches to usability evaluation, in practice the selection of a specific method is often based on considerations of costs and available resources (Hartson et al., 2003;Law et al., 2009).
Drawing a conclusion on which method is best is difficult if not impossible task, since each method has its strengths and drawbacks and there is no specific method for evaluating e-learning systems.A case study on comparative usability evaluation (Koutsabasis et al., 2007) revealed that "no method was found to be significantly more effective or consistent that others".
Methods of usability evaluation are usually very expensive, time-consuming and their results are usually very difficult to analyze.
From these reasons, there is a need to develop a model of usability evaluation that is: − quick, precise and produces results that might be easily analyzed, − allowing to get single-value score, − able to deal with the users' language which is full of vague terms, − based on mathematical principles.

The Advantages of Fuzzy logic
The approach to usability evaluation based on fuzzy logic are used for Human-Centered Systems (Nunes, 2010), web-sites (Gülçin Büyüközkan et al., 2010), web-portals (Hub et al., 2010), e-learning system (Lanzilotti et al., 2010), quality of education (Valdés-Pasarón et al., 2011).We propose to apply this approach to usability evaluation e-learning modules.
Following principles should be implemented in the model of fuzzy usability evaluation (Hub et al., 2010): − users do not express the overall score by using numerical values; − using their natural language, they evaluate a set of characteristic features that significantly affect usability; − users' mental load should be minimized so they can fully focus on the aspects of evaluation.Usability score is a best approximation of expert knowledge stored in a special database.

Problem Solution
According to the methodology of the ergonomic expertise (Adamenko, 1993) we'll solve the problem as follows: 1. Determination evaluation criteria.2. Screening of options, in which at least one ergonomic factor has a value below a certain critical admissibility.3. Determination of the integral index of ergonomic performance of e-learning modules according to set of local indicators.Steps 1 and 2 are adequately described in the ergonomic literature (Adamenko et al., 1993;Rothstein, 1999).The problem of the Stage 3 refers to the classification problem, which can be solved with the involvement of a large number of methods allowing working with peer reviews (method of hierarchy analysis, neural networks, fuzzy logic et al.).Due to the vagueness of the information, contained in the estimates of experts, we define the fuzzy inference method, proposed by Rothstein (1999).
The usability evaluation process is performed by applying the following steps (Fig. 1).

Determination of E-Learning Module Evaluation Criteria
The degree of study of various factors in literature is investigated in the review Bernérus et al. (2010).As a result, looking at Fig. 2, we can see that the top factors are general issues of usability, such as Navigation, Feedback and User Control.However, it is important to note that some general usability factors are more or less important due to the fact that an e-learning system is being evaluated.For example the Flexibility and Efficiency of Use as well as Visibility of System Status are particularly important for e-learning systems, which can also be seen in the result.
Various studies show the validity of the use of different criteria.So, in the work of Gülçin Büyüközkan (2010) the author offers the following global criteria: Right and Understandable Content, Complete Content, Personalization, Security, Navigation, Interactivity, User Interface.Costabile (2005) has identified the following groups of criteria: Presentation, Hypermediality, Application Proactivity, User Activity, Scaffolding, Learning Window.Each criterion has a number of attributes.For example, the criterion Learning Window has attributes of organization of a course document, suitability of formats of course document, check of assessment test presence, check of usage of communication tools, adequacy of learning tools.
When talking about an e-learning system, we do not distinguish between platform and didactic module.Actually, an e-learning platform is a more or less complex environment with a number of integrated tools and services for teaching, learning, communicating and managing learning material.On the other hand, the didactic module is the educational content provided through the platform (the container).Usability attributes for a platform generally differ from those of a specific didactic module (content) (Costabile et al., 2005).The analysis of a number of works allowed to identify 14 basic categories of criteria for e-learning modules evaluation.This number determines the high labor intensity for the evaluation, therefore it is necessary to identify the most important criteria for the general ergonomics.
For an example of the approach, as a basis, we take a list of criteria categories (Table 1) from the review Bernérus et al. (2010).

Determination of Criteria Weights for E-Learning Module Evaluation and Creation
a Criteria Set

Initial Conditions
The task of assessing the significance of the criteria can not be solved only once.The results of this analysis depend on the conditions of the school, the learning process, the contingent of students and other factors.The complexity of this step is not only in the organization of the expertise, but also in the selection of the qualified experts who know the subject area.The results of such expertise in different universities vary.
For conditions of Sumy State University such examination was organized involving 3 experts.Error management This concerns help and documentation files and how easy it is to find the relevant information.
Learn-ability How easy the system is to learn.How long does it take for a user to master the system?

Cognition facilitation, recognition & memorability
Relevant objects, actions and options should be clear.The user should not have to remember too much itself.

Flexibility and efficiency of use
This criteria concerns the possibility for the system to adopt to different users with different learning styles and tastes.

GUI
This concern not only how "pretty" the interface is but also how logical the structure is and how easy it is to read and understand.
Navigation and exiting How easy the system is to navigate and find your way in it.How logical the structure is, etc.

Match between system and the real world
This issue is specifically concerned with logical metaphors and phrases, etc.

Consistency and standards
The learner experiences the user interface as consistent (in control, color, typography, and dialog design) words and concepts that the user can recognise and understand.

Help and documentation
Help should be easy to search.Any provided help is focused on the learner's task, and lists simple concrete steps to be carried out.

Accessibility
How the software can be accessed.All repository access to both teacher and learner.
Learning content design Concerns pedagogical aspects in learning/course materials.Terminology, layout and media use are some examples from the papers.
Visibility of system status Concerns system feedback and to the user, both in terms of system status, presenting a score or other information and similar.

User Control and freedom
The learner can easily turn the application on and off, and can save his user profile in different states.
There are many methods that can be employed to determine weights, such as eigenvector, weighted least square, entropy methods and diverse MCDM methods.In our task of estimating the ergonomics of the module a subjective notion of comfort is considered.Same module with fixed values of the parameters can be interpreted in different ways by different experts and, respectively, the degree of comfort will be different.Therefore, it is necessary to use methods based on subjective assessments of experts.In this study, the most outstanding MCDM approach, AHP (Saaty, 1980) is used to determine the decision criteria weights.
The four step computational procedure is given as follows: Step 1. Compare the performance score.The numbers (1, 3, 5, 7, 9) are used to indicate the relative strength of each pair of elements.
Step 2. Construct the comparison matrix.
Step 3. The consistency ratio (CR) for each matrix is calculated.The deviations from The consistency ratio is used to estimate directly the consistency of pair wise comparisons.If CR is less than 0.10, the comparisons are acceptable, otherwise not (Saaty, 1980).
Step 4. The priority weight of each criterion can be obtained by multiplying the matrix of evaluation ratings by the vector of attribute weights and summing over all attributes.The obtained results are shown in Table 2.

Using the Results of the Criteria Importance Evaluation
The significance of the criteria is important at: − assessment of the modules (primarily the most important criteria are evaluated that allows to postpone a subsequent evaluation of the most important indicators at their extremely low values); − exclusion from examination the least important criteria (in case of restrictions because of the complexity of expertise).In the next section, as an example, a model for the examination of the most significant groups of five criteria is considered: Flexibility and efficiency of use, GUI, Navigation and exiting, Consistency and standards, Learning content design.

A Model of Ergonomic Expertise Based on Fuzzy Logic
Suppose we are given a set of local indicators of ergonomic performance of modules K = {k j }, j = 1, n. Selected indicators of the sets can be separated into some categories G = {g i }, i = 1, m.There is some procedure of conformity evaluation of local indicators to some requirements.Given is a set of possible outcomes of usability evalua-tion E = {e 1 , e 2 , e 3 }.Evaluation of the quality of the module is used to make one of the following decisions: e 1 -conform (module corresponds to the claimed usability recommendations and standards), e 2 -redesign (module not fully corresponds to the claimed usability recommendations and standards), e 3 -mismatch (module dose not corresponds to the claimed usability recommendations and standards).
A set of specific parameters, analyzed in each case, depends on many factors.In this example for ergonomic examination we restrict ourselves to the following characteristics: Navigation and Exiting, Learning Content Design, Consistency and Standards, GUI and Flexibility and Efficiency of Use.This example is illustrative.In reality, reducing the number of groups of factors can be held in conjunction with the need to reduce the complexity of the examination.This decision is made every time based on the analysis of the available time resource of the experts.
Let E -an integral indicator of the quality of e-learning modules.To evaluate this indicator we will use the following information: X -Navigation and Exiting, which is evaluated with the following local indicators: x 1 -usability of the keyboard and mouse, x 2 -intelligibility and easy to navigate (type and location of control keys, key transition points and the path to them, the possibility of random and sequential movement of material, etc.), x 3 -usability of the table of contents; Y -Learning Content Design, which is estimated with the following local indicators: y 1 -the amount (dosage) of the material on the page (slide), y 2 -consistency of page layout; Z -Consistency and Standards, which is estimated with the following local indicators: z 1 -readability, z 2 -compliance with the design logic of the text elements (body text, headings, subheadings, captions for illustrations, etc.); V -Flexibility and Efficiency of Use; M -GUI (compliance with the design logic of the objects): m 1 -ratio aspect, m 2location, m 3 -color.
The task of the evaluation is to bring to conformity one of the solutions e 1 , e 2 , e 3 with the module with known local indicators.

The Scheme of the Solution
The general scheme of solving the problem of ergonomic examination of e-learning modules is shown in Fig. 3 and represents a sequence of actions: 1. Evaluation of the e-learning module according to the selected indicators on the scale of the thermometer.2. The procedure of fuzzy inference.3. The decision on the conformity of the achieved quality indicators to general and special ergonomic requirements and the establishment of the ergonomic quality of the module.If you decide on the conformity, the module is added to the library e-learning modules and can be used for further procedures to select the most suitable module for a particular user.Otherwise, the recommendations are given for redesign, or discrepancy of the module is justified.

Fuzzy Inference
The criteria hierarchy is shown in Fig. 4 as a derivation tree, which corresponds to the system of relations: These relations are brought into line with fuzzy logic equations, which allow to define the level of index E according to the maximum of membership function: In accordance with Rothstein (1999), the algorithm of fuzzy inference using a generalized derivation tree is: 1. We fix a vector of values of input variables We determine the values of membership functions of terms-estimates of input variables μ x jp j (x i ), p = 1, l, j = 1, e j , μ y jp j (y i ), p = 1, m, j = 1, g j , μ z jp i (z i ), i = 1, n. 3. Using ( 6)-( 11 (12)

Evaluation of Local Indicators on the Basis of the Thermometer
The peculiarity of the local indicators is that they are qualitative in nature, ie do not have precise quantitative measurements.Therefore, when evaluating the same figure some experts may have different opinions.In addition the expert is not always able to estimate verbally the local indicator, although intuitively feel the quality.To overcome these difficulties the local indicators can be estimated on the basis of the thermometer (Fig. 5), which is set out in the (Rothstein, 1999) work.The beauty of this approach is that different in the sense local indicators are determined as linguistic variables defined on a single universal set, which is the scale of the

There is no table of contents
In the table of contents there are only the main sections Some inconvenience during the transition

Smooth transition between sections and subsections
The amount of material on the y 1 The redundancy of the material on the page Some redundancy Some comments A necessary and sufficient amount of material Uniformity of pages' design y 2

Unreasonable stylistic diversity
Differences in the design of some pages Minor comments All pages are in the same style thermometer.Subjectivity reducing can be achieved through the use of the recommendations in Table 3.For the shaded portion of the scale in Fig. 5 for the "build quality" corresponds to the high quality of performance.

Evaluation of the Integral Indicator
Suppose that the linguistic variables x 1 −x 3 , y 1 −y 2 , z 1 −z 3 , v, m 1 −m 3 are valued by fuzzy terms L -low, M -middle and H -high, which are defined using the membership functions (Fig. 6).
Using fuzzy terms, we define a knowledge of the relations (1-5) in a matrix, fragments of which are presented in Tables 4-8.Each group of lines shows a conditional statement that links the fuzzy values of the input and output variables.For example, Table 4 shows that a statement is the condition of the matching module: A fragment of knowledge of the relations (1) Fuzzy logic equations, set out in compliance with Tables 4-8, allow to evaluate the integral coefficient of agreement for fixed values of local indicators.Using the above approach, we form a set of modules that the best meet the requirements.Examples of estimates of the three modules are shown in Table 9.

Automation of the Examination and Analysis of the Approach Benefits
In Sumy State University examination is carried out using this approach by means of MatLab.Basic calculations are performed using Fuzzy Logic Toolbox.This solution has Table 9 Examples of e-learning usability evaluation Fig. 7. Relations between ergonomic quality of e-learning modules, learning content design and GUI. the following advantages: − no need to purchase (to develop) expensive programs; − ease of creating and editing the necessary knowledge bases; − no time-consuming calculations; − rapid assessment; − ability to solve problems, such as "What if?"; − focusing on untrained users, user-friendly interface; − large graphical representation of data (see example in Fig. 7); − ability to connect other tasks (for example, adaptation to the requirements of students).
Some of the automation matters of such assessment are considered by Lavrov and Barchenko (2008).

Conclusions
The large number of existing scientific and methodological literature on quality assessment of e-learning resource demonstrates the relevance of the theme for the educational sector, diversity of approaches, methods and assessment tools, The complexity of most methodologies of expertise and lack of available decision support systems in the formation of repositories of educational resources make the task of developing a procedure for resource evaluation of current importance.
For the examination of the quality we offer the approach based on the use of fuzzy logic, which allows you to: − create simple intuitive expert estimation procedures of local indicators of quality; − obtain the integral quality assessment; − work out recommendations for the development of a repository of educational resources.The developed approach is tested in the ergonomic expertise of educational resources for distance learning system of Sumy State University.
The use of models and derived from them automated procedures can significantly reduce the complexity of evaluation of educational resources.
Further studies will be aimed at improving the software tools and techniques for evaluation of e-learning modules within the project "Mechanisms of the organization of the individual educational trajectories using electronic resources to improve the quality of training", which is scheduled for implementation.

Fig. 3 .
Fig. 3.The general scheme of solving the problem of ergonomic examination of e-learning modules.

Fig. 5 .
Fig. 5. Evaluation of the variable u on the principle of the thermometer.
), we calculate the membership function of the terms-estimates of the output value E, which corresponds to the vector of values of input variables

Table 3
Some recommendations to assessment of local quality indicators