An Automatic and Dynamic Approach for Personalized Recommendation of Learning Objects Considering Students Learning Styles : An Experimental Analysis

Content personalization in educational systems is an increasing research area. Studies show that students tend to have better performances when the content is customized according to his/her preferences. One important aspect of students particularities is how they prefer to learn. In this context, students learning styles should be considered, due to the importance of this feature to the adaptivity process in such systems. Thus, this work presents an efficient approach for personalization of the teaching process based on learning styles. Our approach is based on an expert system that implements a set of rules which classifies learning objects according to their teaching style, and then automatically filters learning objects according to students’ learning styles. The best adapted learning objects are ranked and recommended to the student. Preliminary experiments suggest promising results.


Introduction
The evolution of technology has allowed teachers and students to use the computer as a teaching-learning resource.Currently, it is possible that teachers, even without advanced computer skills, provide digital lessons.However, classic distance education environments, in which prevail the traditional teaching model (the same content presented for all students), have, according to Brooks et al. (2006), several limitations.Therefore, it is interesting to consider the particularities of each student, who has his/her own preferences in relation to learning and, therefore, tends to have a better experience of use, when the presentation of content is targeted to his/her student's profile.In this context, approaches for content adaptation in accordance with particular students characteristics have been proposed (Nat et al., 2010).As a consequence, the personalization in learning environments has been widely investigated.The main goal of this personalization is to provide more relevant content and information to students.
In addition, technological advances have led to progress in the area of Informatics in Education.The incorporation of Learning Objects (LO), defined by the IEEE (IEEE, 2010), and its metadata standard for Learning Objects (LOM -Learning Object Metadata), contributed to this progress.LO are entities -digital or non-digital -which can be used and reused for teaching, education or training.
With respect to educational theories, Felder and Silverman (1988) developed a Learning Styles (LS) model based on four dimensions, each one with two poles.The student preference can be considered balanced for a dimension if there is no clear trend to a pole.Also, it can be considered moderate or strong, if there is clear trend of preference for a specific LS.The goal of the research conducted by Felder and Silverman (1988) was to compare the most common profiles among students and teachers' teaching mode.The main characteristics of this model are presented forward in this work.
According to Kinshuk et al. (2009), the field of LS is complex and many questions are still open, including a clear definition of LS.Investigations have been conducted on relating LS characteristics to LO characteristics.In order to provide adaptivity, students LS have to be known firstly.Therefore, a consistent student model is an important issue in adaptive educational systems.
The high difficulty in manually obtaining the best suited LO to each student, in any repository, is based on the fact that there is a wide range of different students with different preferences.This condition can be even more striking when considering LO repositories that have a high number of objects to be scanned.In this context, the need for an effective mechanism to perform automatic correlation between LS and LO is clear.
Thus, this paper presents a practical approach to effective and automatic recommendation of LO according to LS, considering the model of Felder and Silverman.This work considers the IEEE LOM, and uses it to perform personalized recovering and recommendation of LO according to specific students LS.
For this, we show that LO delivery, given a student model that define specific learning preferences, can be automated.The main motivation for this work is the impossibility to recover and manually recommend LO that best fit each specific student, due to the wide variation of student profiles, and the large number of LO that must be analysed.As a result, we obtained an automatic and dynamic approach for personalized delivery of LO, considering students LS.
To this end, a detailed analysis was carried out on the properties of each LS in the model proposed and described by Felder and Silverman.Furthermore, a detailed study of IEEE LOM, including its fields and properties, has also been done, making it possible to find out which LOM fields could bring information related to LS in the Felder and Silverman model.From this, an expert system for classification of LO according to LS was implemented, allowing the development of a content recommendation system according to students LS.
The remainder of this paper is organized as follows: next section presents the main concepts related to our approach.Then, we present and discuss some related work.Continuing, we present in detail our recommendation approach.After that, we show some results obtained from experiments.Finally, it is presented our conclusions, contributions and future work.

Theoretical Background
In this section, it is presented the definitions and concepts necessary for the understanding of the proposed approach.

Learning Styles and Student Modeling
A learning styles model classifies students in accordance to the way they perceive and processes information.A learning style can determine how an individual interacts and reacts in a learning environment, reflecting their real preferences.There are several models that describe ways of classifying a student in a particular learning style (Felder and Brent, 2005).Many learning styles models have been proposed, for example, (Entwistle, 1981), (Felder and Silverman, 1988), (Honey and Mumford, 1992), (Kolb et al., 1984) and (Pask, 1976).Each one of these models describe different aspects on how students prefer to learn.Felder and Silverman (1988) present a theoretical model in which each student can be classified into different LS in four dimensions.In this way, each student can adopt attitudes and behaviours that are repeated at different times and situations.To meet the different learning styles is necessary to use appropriate teaching strategies related to various perspectives of learning.In short, LS incorporate individual features for tasks like organize, understand, process, remember and thinking in order to learn and solve a problem.
A striking feature of the Felder and Silverman model is that while most models classify students in fixed types, Felder and Silverman's model is based on the idea that each student has a tendency to one style in each one of four dimensions.The following presents a brief description of the dimensions and learning styles that make up this model (Felder and Silverman, 1988) According to Thompson (1996), the main feature of an adaptive educational system is the ability of personalization.In this context, Souza (2002) states that the student model is considered a critical piece of individualized behaviour in this type of system, which strongly depends on how the knowledge about the student is modelled internally.
Therefore, the student modelling is a critical issue in developing this type of system, thus requiring special attention.One of the problems in these systems is to determine and represent effectively and consistently the information about students, as alert Thompson (1996).
Nevertheless, student modelling can be classified into static or dynamic (Graf et al., 2010).Student static modelling refers to an approach in which the student model is initialized only once, usually when the student enrols in the system.On the other hand, the dynamic approach to student modelling often updates the student information, and then allows the system to respond to changes in the student model during the course (Graf et al., 2010).Our approach considers a dynamic student modelling, as exposed in Section 4.

Learning Objects and IEEE-LOM
Web based teaching-learning environments are increasingly being used to support education activities.The development of educational materials in these environments requires creativity, time availability and knowledge on the appropriate technologies.In order to minimize the time and effort spent on the development of educational material -and thinking of a way to reuse this material -the concept of Learning Objects (LO) is a natural choice.
The IEEE defines LO as entities, digital or non digital, which can be used for teaching.Other authors complement this setting.According to Wiley (2000), LO are digital resources designed to support teaching and enable reusing.With the growing demand for these types of resources, it was necessary to stipulate a standard for the identification and reuse of them.In this sense, organizations like IEEE, IMS Global Learning Consortium and ADL (Advanced Distributed Learning), produced specifications and standards to address these needs.
The IEEE metadata standard for LO, which is called the LOM (Learning Object Metadata), is a standard that describes LO in accordance with a commonly accepted model, which facilitates interoperability between different content repositories and learning systems.This pattern contains a conceptual data schema that defines the structure of metadata for LO, which is usually encoded in XML.
Metadata are information about a learning object, whether physical or digital.As the supply of LO is growing, the lack of information or metadata about LO occupies a critical and fundamental point in the ability of discovering, managing and using them (IEEE, 2010).Metadata allows automating the search and recovery of content, which can be reused in different learning scopes and systems.They define the minimum set of properties required to allow the management, location and evaluation of these objects, used to contextualize the characteristics of a particular item of information.
IEEE LOM brings a diversity of categories of data fields.The Educational category was of fundamental importance for this work, just to focus the description of pedagogical features of learning objects.
Among these works, Romero et al. (2007) present a component that suggests links to customize the user interaction based on data mining algorithms that evaluate the student's log.The component was integrated to the project AHA!, a virtual learning environment based on adaptive hypermedia for offering content to students.Recommendations are made based on registered links on the course.However, the relationships between LO and student profiles are not carried out in an automated manner, requiring a tutor to manually perform the correlation at the registration of objects.
The work presented by Milosevic et al. (2007) builds learning scenarios according to Kolb's model (Kolb et al., 1984), and associates students LS to LO metadata considering only Learning Resource Type and Semantic Density fields from LOM's Educational category.
The system presented by da Silva and Rosatelli ( 2006) was designed to be used in distance learning courses on the web.Therefore, it performs the recommendation of content based on knowledge (beginner, intermediate or advanced).The system monitors the activities performed by the student and classifies the student.It is from this that the classification system performs the adjustment of navigation on employing the concealment techniques and annotation links.There is no automatic association between the links and the learning styles.
It is important to say that some studies perform analysis on what types of electronic media are more suitable to a particular LS (Franzoni and Assar, 2009).Lopes and Fernandes (2009) observed that some instructional actions are more appropriate for a learning style than others.It was assigned values in the interval [0,1] to actions, which indicates whether an action satisfies a learning style.The work presents an example of this association policy between instructional actions and LS.Yet, Sangineto et al. (2008) classifies different types of LO according to the value of the attribute Learning Resource Type, from the IMS Metadata Standard.Bittencourt and Costa (2011) present some models and tools for building adaptive and semantic educational systems, describing a multilayer architecture and a set of ontologies and mapping rules to be used in the construction of this type of system.Also, Candotti et al. (2006) shows the implementation of a display module adapted according to students learning styles, generating different materials according to their profiles.Zaina et al. (2012) proposed an approach that considers the LOM fields Interactivity Type and Learning Resource Type to make recommendations.Besides, the authors consider 3 FSLSM dimensions: Perception, Input and Processing.The recommended LO have equivalent weights.
In addition, systems such as AHA! (De Bra et al., 2006), KOD (Manouselis and Sampson, 2002), LSAs (Bajraktarevic et al., 2003), TANGOW (Paredes and Rodriguez, 2004), LS-Plan (Limongelli et al., 2009), Manic citepstern2000adaptive, among others, have the main focus on providing a learning experience that meets students learning styles.For this, they use self-assessment questionnaires, as Index of Learning Styles Questionnaire (ILS) (Van Zwanenberg et al., 2000), inferring students learning styles without any, or with low level of automation.
It is important to mention that it was observed in the analysed related work that they consider only a subset of LS.Generally, they do not use a metadata standard for LO, or they don't consider a well-known LS model, or both.Therefore, the personalization process provided by these approaches must be ineffective.Thus, progress on these aspects is a major contribution of this work, which considers all dimensions of Felder and Silverman LS model, and considers the IEEE LOM as metadata standard for LO.
Finally, the automatic recovery of LO on the web, through their metadata, taking into account specific features of LS, is an important research field.Large-scale re-use of LO and interoperability in virtual learning environments have been made possible by technologies of the SemanticWeb (Davies, 2006).SemanticWeb has its basis in eXtensible Markup Language (XML) (W3C, 2010a) and standards for knowledge representation such as the Re-source Description Framework (RDF) (W3C, 2010b) and the Ontology Web Language (OWL) (W3C, 2010c).The search and manipulation of this interchangeable knowledge is facilitated by query languages such as XQuery (W3C, 2010d).

Content Recommendation Based on Learning Styles
Our approach is based on a probabilistic student model, in which LS are automatically discovered and evolve over time, as presented by Dorça et al. (2013).If a self-assessment questionnaire is used for initialization of the model, as ILS (Felder and Spurlin, 2005), the probabilistic student model can be booted from the data obtained by the questionnaire, considering the proportion of responses scored for each LS inside a dimension.If no selfassessment questionnaire is used, the probabilistic student model is initialized with 0.50 (cold start -undefined preferences).
Therefore, in the approach presented by Dorça et al. (2013), LS are considered as probabilities, and not as certainties.Table 1 shows an example of a probabilistic student LS model, according to the approach proposed by Dorça et al. (2013).In this example, the numbers represent the probabilities of preference for each LS in a dimension.Thus, in the example, the student has 35% probability to tend to the active style, and 65% probability to tend to the reflective style.A detailed explanation on how this model is built can be obtained in (Dorça et al., 2013).This probabilistic model is built from a stochastic process, which is used as basis for recommendation of LO on the proposal presented in this work.
The main goal is to automatically adapt the teaching process for each student according to LS.In this context, it is observed that automatic recovery of LO in repositories is a critical issue for the personalization of the teaching process according to LS.
For this, the development of this approach followed the following working methodology: Study and analysis of the properties of learning styles, considering the theory of 1.
Felder and Silverman.Study and analysis of metadata fields and values of the IEEE LOM standard.

2.
Definition of which LOM fields allow the classification of a learning object ac-3.
cording to learning styles in the Felder and Silverman's model.Mapping of the relation between these fields and the learning styles in the Felder 4.
and Silverman model.Implementation of an expert system for classification of LO according to LS, using 5.
the knowledge obtained in the previous step.Implementation of a recommender system for automatic recommendation of LO 6.
according to students LS, taking into account the LO relevance order for a specific student.Implementation of a prototype for experimentation with the proposed approach.7.
In this context, Table 2 presents the relations found between LS and LOM, showing, in each cell, which LS is satisfied by a specific value of LOM.This enabled to create a system that finds and recommends the best fit LO for a specific student.
These relations have been found from careful study and analysis of the following basis documents: (Felder and Silverman, 1988) and (IEEE, 2010).The first one is clear on presenting the main characteristics and preferences of each kind of student, and how they prefer to learn.The second one gives details on the LOM fields and its values, and provides explanations on the semantics of the metadata.Therefore, these two publica- It is important to notice that a subset of LOM is being used.We have used a minimal subset of LOM, which provides sufficient information about a LO in order to recommend it to a student.This subset is enough to characterize a LO according to its teaching style, enabling the system to recommend them appropriately to students.In order to properly recommend a LO according to students' LS, it is necessary to know, at least, its structure, format, interactivity type, learning resource type and interactivity level.We have concluded this based on the analysis of (Felder and Silverman, 1988) and (IEEE, 2010).
The following four paragraphs summarize the analysis that we proceeded, and explains the origin of the relations presented on Table 2.
According to Table 2, sensitive learners must be attended by videos, applications, simulations, graphs, indexes, tables and experiments.This is because this kind of LO surely brings content addressed more concretely (facts, data and experimentation), which is preferred by this kind of student, as stated by Felder and Silverman (1988).On the other hand, Intuitive learners have greater preference for more abstract content (principles, theories, mathematical models) (Felder and Silverman, 1988).As we know, diagrams are usually used to represent abstractions of reality.In this way, this kind of LO is considered to be more appropriate for intuitive learners.
Active learning (e.g., learning by doing) is supported by content that directly induces productive action by the learner (Felder and Silverman, 1988).Active documents include exercises, simulations, questionnaires, exams, experiments, problem statements and self assessments.Reflective learning (e.g., passive learning) occurs when the learner's job mainly consists of absorbing the content exposed to him (generally through text, images or sound).An expositive learning object displays information but does not prompt the learner for any semantically meaningful input.Expositive documents include essays, video clips, all kinds of graphical material, and hypertext documents (IEEE, 2010).Therefore, LO with high interactivity level are considered to be better adapted to active learners, while LO with low interactivity level are better adapted to reflective learners.
A student may have preference for information transmitted visually, considered to be a visual student, or information provided as text or speech, and in this case the student is classified as verbal, as stated by Felder and Silverman (1988).Therefore, we defined that material with dominant visual and graphical content, as image, diagram, figure, graph and experiments are proper for visual students.Material which have more textual content, as audio, text, forms, questionnaires, among others, are proper for verbal students.Videos and applications commonly mix verbal and visual content.Therefore, they can be recommended for visual or verbal students.
Global students prefer LO with networked structure, and sequential learners prefer LO with linear structure (Felder and Silverman, 1988).As sequential learners prefer content displayed in a progressive way and more restricted view, Table 2 considers LO with sequential structure better adapted to this kind of student.In this context, slides are considered to be better adapted to sequential learners, due to its usual linear presentation.
Using the relations discovered from this analysis, and presented in Table 2, an expert system for classification of LO according to LS based on (Felder and Silverman, 1988) and (IEEE, 2010) have been implemented.The expert system is composed by a set of rules.Two of those rules are following presented.These rules are related to the classification of a learning object in respect to its interactivity level.The rules are written in a PROLOG like language, and they are presented by Fig. 1.
Fig. 1 presents the set of rules divided in five sub-sets, according to the LOM field that they analyse: Structure, Format, Interactivity Type, Learning Resource Type and Interactivity Level.
For example, the rules in the sub-set Interactivity Level state that a learning object X is adherent to a reflective student if it has interactivity level very low, low, or medium.And, that a learning object X is adherent to a active student if it has interactivity level medium, or high, or very high.The rules implement exactly the knowledge presented in Table 2.
For each rule satisfied by a learning object, its relevance is increased by the corresponding value stored in the probabilistic LS model.For example, considering the LS model presented in Table 1, if a LO X has medium interactivity level, then it satisfies both rules presented above, and consequently, its relevance is increased by 65, considering that the student has a tendency of 65% to the reflective learning style.In addition, its relevance is increased by 35, considering that the student has a tendency of 35% to the active style.But, a LO with high interactivity level should have its relevance increased only by 35, considering that only the second rule would be satisfied.Therefore, the system sums the values obtained by a learning object in each of the eight learning styles.Thus, the calculation of relevance R for a LO is given by (1).In (1), Q i is the number of rules satisfied by the LO, considering a specific LS.LS i represents the probability value stored in the student model for the LS i.Therefore, the index i ranges from 1 to 8.

(
) From this, the system automatically recommends LO that have some level of relevance to the user profile.The end result is a list of LO, sorted by relevance, considering a specific student's LS.The most relevant appear on the top of the list.The next section presents some experiments and results.

Experiments and Results
For execution of experiments, a LO repository was simulated through a relational database.Experiments were performed using the probabilistic LS model, as proposed by Dorça et al. (2013).For all experiments, the subject of study was Sciences and Arts.The prototype for experimentation is shown in Fig. 2. The prototype allows the user to performing up the entry of the student's probabilistic LS, and shows the recommended LO sorted by relevance.

Experiment 1
In this experiment, we considered the following probabilistic values for LS: Sensitive 0.70; Intuitive 0.30; Visual 0.55; Verbal 0.45; Active 0.1; Reflective 0.9; Sequential 0.8; Global 0.2.Fig. 3 presents the result -the learning objects recommended for this profile, duly ordained by relevance according to the student's LS.
As we can observe, the learning object with code 8 was the most adherent to the student's LS.This LO has a relevance value of 510 according to (1).It is possible to notice that considering the LS visual, the field Format is met (video), and adds 55 to the relevance of this learning object.For the verbal style, the value video in the field Format increases the relevance of the LO by 45 points.Considering the active style, the relevance is increased by 10, due to the field Resource Type, which has the value simulation.Relevance is analysed and accounted for each LS.The same process is performed for the other LO, resulting on the list presented.

Experiment 2
The student model used in this experiment is: Sensitive 0.70; Intuitive 0.30; Visual 0.55; Verbal 0.45; Active 0.1; Reflective 0.9; Sequential 0.2; Global 0.8.In this experiment, the student model has the value global greater than sequential.
This change in the student profile affected the ordering of LO in relation to experiment 1, as shown in Fig. 4. In this experiment, the LO that most meets the student's LS is the one with code 7. What leads to this fact is the field Structure with value networked present in this learning object, which is matched by the student's preference for the global style, increasing by 80 the relevance of this learning object.
The learning object with code 8, which was the most relevant in the previous experiment, appears in fifth.The learning object significantly lost its relevance, due to its linear value for the field Structure, which doesn't meet the supposed student preference.

Experiment 3
For this experiment, comparing to the previous, the student profile has had the values of the processing dimension reversed.Therefore, in this experiment we had the value 0.9 for the active learning style and 0.1 for reflective, which resulted in the following: Sensitive 0.70; Intuitive 0.30; Visual 0.55; Verbal 0.45; Active 0.9; Reflective 0.1; Sequential 0.2; Global 0.8.This change caused the learning object with code 10 to be the most relevant for the student, as shown in Fig. 5.This change in the recommendation is due to the fact that the learning object with code 7 (best recommended in the previous experiment) lost its relevance, by presenting reflective characteristic, and does not meet strongly active students.Thus, the probability of 90% for the active LS in the current experiment causes the learning object with code 10 to be the most relevant.
The results obtained in this section allows us to conclude that the presented approach satisfies the goals originally proposed, and can be deployed and used in existing LMS (Learning Management Systems).The experiments resulted on effective personalized recommendation, taking into account the student's learning styles with probabilistic values.As students probabilistic learning styles do not remain constant over time (they are updated automatically and dynamically throughout the teaching-learning process), it is possible to notice that this process has a non-deterministic feature.
The validation of the proposed approach through a prototype for tests, in which we can enter the student's LS freely, was essential for the completion of this work, considering how much resource and time should be required for the implementation and test of it in a e-learning system.
Therefore, it is necessary to evaluate models before using it in a LMS, due to the complexity involved in this type of project.This need becomes even more evident when analysing the demand for this type of resource (Graf and Kinshuk, 2010).The next section presents some conclusions and future works.

Conclusions, Contributions and Future Work
This paper presents a practical approach to effective and automatic recommendation of LO according to LS, considering the model of Felder and Silverman.This work considered the IEEE LOM, and used it to perform personalized recommendation of LO according to specific students LS.
The main contribution of this work is to achieve an efficient approach for mapping characteristics of LO to students' LS dynamically and automatically, in order to provide personalized teaching process.This demonstrates the possibility of automating the linking between LO and LS when using the IEEE LOM and the Felder and Silverman LS model (Felder and Silverman, 1988).
In this context, a central contribution of this work is the set of rules presented in section 4, which may be easily implemented in any programming language, and used to provide adaptation in any learning environment.
Another important contribution is the LO ranking process, which uses a relevance calculation method presented in section 4. Ranking LO according to LS is an important issue, considering that, in most of the times, more than one LO can be used to present the same content.
As a result, a rule based expert system has been implemented, and this paper depicts its functioning and presents the complete set of rules of the recommendation system.In addition, an important result is the ordering of LO according to their relevance for a specific student profile.The probabilistic modelling of LS used in this work was of fundamental importance for this result.
The main challenge faced was to establish the relationship between LS and LOM, presented by Table 2.This mapping demanded much research and time, requiring study and comprehensive analysis of large content.
The prototype used for experiments, presented in section 5, was of fundamental importance to the work, because it allowed us to easily observe, in details, how the proposed approach behaves in different situations.
As possible future work, it is being analysed the use of more IEEE LOM fields for filtering learning objects.In this way, we have the intention to consider other aspects than LS for recommending content.
In a second step, this work will be deployed on an existing LMS, generating the probabilistic student model and returning the most relevant learning objects for real students.Therefore, will be held evaluating the appropriateness of the proposed model for the interests of real students, the possibility of inconsistencies between the recommended learning objects and what they are looking for at one point of their learning process.

Fig. 3 :
Fig. 3: Recommendation of LO resulted from the first experiment.

Fig. 4 :
Fig. 4: Recommendation of LO resulted from the second experiment.

Fig. 5 :
Fig. 5: Recommendation of LO resulted from the third experiment.

Table 1
Probabilistic student model in which learning styles are treated as probabilities