Abstract
A web-based learning portal offering individualized learning was developed by utilizing rule-based knowledge representation and artificial intelligence techniques of expert systems in order to reduce the uncertainties of learning to minimum and to construct an intelligent tutoring system. This portal offers individualized learning content based on the individual’s level of cognitive knowledge. In order to determine the effects of the developed system on the student achievement, the system was tested in an 8-week-long study on the students of Software Engineering Department of Technology Faculty. The pretest-posttest control group experimental design was used in the study. The experimental group received education with Individualized Instruction Portal while the control group received education in traditional learning environment. Academic achievement test was used as the data collection tool. In order to test the research hypotheses, data obtained from the data collection tools were analysed in terms of frequency, percentages, and dependent-independent -test with statistical software program. Based on the results, no significant differences were found between the groups in terms of the pretest. On the other hand, significant differences were found between experimental and control group in terms of the posttest. It was concluded that individualized learning portal had positive effect on the students’ learning when used in combination with traditional learning environment.
1. Introduction
In recent years, rapidly developing computer technology has become a sector whose use area is expanding everyday [1]. For this reason, computer technology has been used in the field of education in various applications. Use of technology has played an important role in developing teaching and learning processes. On the other hand, information has become crucial for the economic development of developed societies. The rapid development in information technology has contributed to the occurrence of information societies and these societies are forced to follow and adopt the technological developments to maintain their presence. The increase in the number of students and amount of information has brought along some problems. Technological developments playing a key role in the improvement of the quality of education should become a part of educational institutions [2, 3].
The system in which teaching takes place with the use of a software system by locating students at the center of the teaching process is called Computer Assisted Learning (CAL). CAL is a teaching system that increases the motivation of the students, strengthens teaching process, and combines computer technology with the principles of auto-learning [2, 4–6].
Experts systems, a developing discipline in recent years, have started to be used in CAL software. These are systems in which computers perform all the things that are expected to be performed by a human expert in a given field through some tools [7]. In line with this idea, a new teaching system “Intelligent Tutoring Systems (ITS)” has appeared under the heading of developed CAL systems. The infrastructure of ITS consisted of the expert information model, user model, and instruction model which was introduced by Hartley and Sleeman in 1973. For nearly thirty years, this model was used and a fourth component which is “User Interface Model” has been added to it [8].
Intelligent Tutoring Systems (ITS). Intelligent Tutoring System (ITS) can be briefly defined as computer programs that know what to teach, how to teach, and whom to teach by using artificial intelligence and instructional technologies. In another definition, ITS is defined as the computer programs that use artificial intelligence techniques while representing the information and providing interaction with students [9, 10].
ITS is the last step of advanced learning technologies and is developed to support CAL. When considering the tasks of ITS and CAL, it can be seen that these are different from each other. The primary reason for this difference is that CAL uses computer as a technology whereas ITS uses the software as a teaching tool.
Although ITS is seen as virtual learning, its effectiveness can be realized when considered the results of researches conducted. According to a study comparing intelligent tutoring systems with traditional computer-aided teaching systems in Carnegie Mellon University, it was determined that ITS increases the quality of learning by 43% and reduces learning time by 30% [11]. It is concluded that one of the main reasons of the increase in the quality of learning is the shift to microeducation, personalization, and the adjustment of the level according to the proficiency level of the students in the teaching process. They compared the questions such as whether the students do the questions, whether the answers of the students to the questions are true, and whether the students do them in the correct order.
ITS enables students to take their personal learning needs into consideration. When these systems are used in combination with the traditional instruction methods, they reduce students’ learning time [12]. Although ITS is a well-established solution and supporter of teaching, they are complex systems hard to set up. In order to set ITS, expert systems, artificial intelligence techniques, and teaching technologies should be used together [13].
One of the most important problems of the education system is the increase in the number of students parallel to the global population growth. Because of this problem, teachers are forced to apply a method whose instructional aims are same and teaching is performed in line with these aims to all the students having different levels, interests, and capabilities. This method in which Individualized Instruction cannot be achieved brings about some problems such as not providing enough feedback and not determining the strengths and weaknesses of the students. In the light of the results of studies conducted to solve these problems, the idea of developing education models in which teaching and learning activities are performed with the computer in an interactive way and teaching role is undertaken by the computer have gained importance.
In many studies conducted and in many educational software programs developed, the usefulness of providing information to students and evaluating and directing them with the support of computers has been sought to be determined. These efforts have led to the new research area called Computer Assisted Instruction (CAI). CAI is the use of computers in teaching a given subject or concept or in reinforcing the gained behavior through the lectures programmed with computers [14].
The developments in artificial intelligence techniques which are important research field of software technology and the researches conducted in CAI have led to the view that software designed in this context is not used adequately and effectively. The supporters of this view recommend that the second type of CAI called ITS should be developed and researches should be conducted in this field [7, 15]. ITS is a software system that can imitate the human teacher in teaching complex subjects and provide individualized learning opportunities for the students [16]. When these systems are used in combination with the traditional instruction methods, they provide the students the opportunity to learn in shorter time and more effectively [17, 18]. In this study, it is thought that investigating the intelligent tutoring systems is important and useful by paying attention to the current education approaches and the effects of the designed “Individualized Instruction Portal” on the academic achievement of students were investigated.
Developed web-based training portal (LessonTutor) for this study consists of information subject which students should know, extraction mechanism, and user interface leading to communication with students and student modeling unit. Active Server Page (ASP) is used for the design and coding of the user interface of components of the system. MySQL is used for the database. The developed system architecture is shown in Figure 1.
Students enroll the system with information like faculty, department, user name, password, and first and last name at beginning. In the second step, students can connect with their own username and password to the system. Then if students’ model was not created before system itself subject to a level test to measure the student’s level of knowledge to create the students’ model, student model is created after determining student placement test level according to the results. The results are recorded in the students’ model and redirected to the page in accordance with the prebuilt model student-generated content or subject. After students study all subjects in the unit, different assessment test for each student held unit is activated by system. If student’s model shows success at the end of the unit test, student’s evaluation units are being updated on an upper unit, being directed to the relevant topics. In case of failure of the student in the unit assessment test system is directed to related topics to eliminate the deficiencies in this unit. For each subject test score is calculated and these values are recorded to the table belonging to students. Students’ model flowchart of the system is shown in Figure 2.
All subjects in the units are shown in one screen in student’s subject page. In this page according to the answers given by the students in placement test and criteria determined by designers, knowledge level of students on each subject was derived. Student subject page is shown in Figure 3.
The student’s achievement level, like which questions were answered accurately, which questions were answered incorrectly, and all issues, can be examined to assess the level of information in the system. This is shown in Figure 4.
In this study, a new model was designed in order to achieve more effective instruction and the effects of this model on the academic achievement of students were investigated. The process was performed in the Introduction to Computer Science lecture. In the development process of this model, it was aimed at increasing the interest and motivation of the students to the lecture by evaluating the needs and the wishes of the students during the education. With this study, the developed system was used to support traditional learning environment. Depending on the general objectives, the subobjective below was determined:
To compare the achievement scores of students receiving education with Individualized Instruction Portal (experimental group) and those receiving education in traditional instruction environments (control group)
There were two groups in the study conducted within the scope of Introduction to Computer Science lecture. One of them was the control group receiving education in traditional education environment and the other was the experimental group receiving education in Individualized Instruction Portal. Academic achievement scores of the two groups were investigated. The following hypotheses were tested in order to reach the subobjectives:(i)There are no significant differences between the pretest and posttest scores of the experimental group receiving education in the Individualized Instruction Portal.(ii)There are no significant differences between the pretest and posttest scores of the control group receiving education in traditional instruction environment.(iii)There are no significant differences between the pretest scores of the experimental group receiving education in the Individualized Instruction Portal and the control group receiving education in traditional instruction environment.(iv)There are no significant differences between the posttest scores of the experimental group receiving education in the Individualized Instruction Portal and the control group receiving education in traditional instruction environment.
The study was conducted depending on the following assumptions:(i)In creating the experimental and control group, the status of the students about whether they have computer and internet connection was considered. Moreover, the demands of students of which learning environment they want to participate in were also considered. The information and skill levels of students related to web-based applications and the equation efforts depending on the pretest scores of experimental and control groups are adequate in terms of eliminating biasedness.(ii)The computers skills and internet capabilities of students are considered adequate in creating the experimental group.(iii)Students sincerely responded to questions relating to the scale of academic achievement.
2. Methodology
In this study it is sought to determine whether the independent variable which is the Individualized Instruction web environment developed affects the dependent variable which is academic achievement. This research is an experimental study. In experimental studies, “it is sought to determine how sysyematic [sic] changes in the independent variable affect dependent variable” [19].
As requirements of the experimental research model, experiment and control group were formed. There were one experimental and one control group in the study. In the creation process of the groups, it was paid special attention that the students in the groups have similar characteristics. Also to minimize the lecturer’s effect to study, the whole study group was conducted by same lecturer and researchers participated just as observers.
The groups and their characteristics are as follows: Experimental Group. The group receiving education in Individualized Instruction portal in addition to traditional learning environment. Control Group. The group receiving education in traditional learning environment with the material prepared as usual.
These two groups were given measurement tools before and after the study in order to determine their academic achievement. The control group were given presentation in the traditional instruction environment and the experimental group were given presentation in the Individualized Instruction Portal for 8 weeks. The study was designed as a “pretest-posttest control group” research model.
Academic achievement scale was used in “Introduction to Computer Science” lecture in order to determine the achievements of Software Engineering Students in the Department of Software Engineering in the Technology Faculty, Firat University, Turkey. The measurement scale used in the study was designed by the researcher and the reliability and validity of the scale were calculated. The development phases of this scale and the calculations related to its reliability and validity are explained below.
“An achievement test” consisting of 47 questions related to units of “Microsoft Windows and Microsoft Word” of “Introduction to Computer Science” lecture in the first semester of the Department of Software Engineering was developed.
The achievement test was used both to determine the entry level of Software Engineering Students at the beginning of the study and to determine the achievement of them after completion of the study. Expert opinions were taken for the achievement test consisting of 70 questions initially in order to determine the content validity of the test and necessary changes were made depending on their opinions. Before the implementation of the test to the groups in the study, in order to ensure validity and reliability, the test was administered to 162 students in total consisting of second (59 students), third (69 students), and fourth graders (34 students) who had taken the lecture before. All the questions in the test were multiple-choice questions each of which is 1 point. After the administration of test, necessary analysis was made. Depending on the item analysis, the difficulty and relevance indexes of each item were calculated. The items which had relevance index less than .20 were removed from the test and the remaining 47 items formed the achievement test. Data related to item analysis can be seen in Table 1.
Item difficulties range from .33 to .90. It can be concluded that the test includes easy and difficult items. The average difficulty of the test was found as .69. When considering the ideal average test difficulty being .50 depending on studies in the literature, the test was accepted as medium difficulty which is “desired average difficulty.” The reliability coefficient of the test was calculated as .75 depending on the KR-20 formulization. When all these results were taken into consideration, it can be concluded that the achievement test is reliable. Statistical calculations and the obtained data can be seen in Table 2.
The achievement tests were applied to 56 students of daytime and evening education of first classes of the Department of Software Engineering of Firat University as a pretest. At the end of the study, the same test was administered to the groups as the posttest.
For the analysis, the obtained data were transformed to SPSS. In the analysis section related to the data of academic achievement test, dependent and independent -tests were used where two variables exist.
3. Findings
In this section of the study, statistical findings related to the subobjectives of the problem status. The achievement test was administered to the students twice as a pre- and posttest. Statistical calculations were made on the scores of students in line with the hypothesis. The findings and comments related to hypothesis are given in this section.
The achievement test was administered to the experimental and control group twice as a pre- and posttest. Statistical calculations were made on the scores of students in line with the hypothesis. Findings and comments on hypothesis related to achievement test are given below.
Hypothesis 1 (there is no significant difference between the pretest and posttest scores of the experimental group). The dependent group -test results of the academic achievement test which was conducted as pre- and posttest to determine whether Individualized Instruction Portal has effects on the academic achievement of the students in the experimental group can be seen in Table 3.
As seen in Table 3, a significant difference at the level of was found between the pretest and posttest scores of the experimental group. The arithmetic mean of the pretest of experimental group was 29.28 and 36.25 for the posttest. According to this result, hypothesis 1 was refuted. This result shows that Individualized Instruction Portal had a positive effect on the academic achievements of the students.
Hypothesis 2 (there is no significant difference between the pretest and posttest scores of the control group). The dependent groups -test results of the academic achievement test which was conducted as pre- and posttest to the control group can be seen in Table 4.
As shown in Table 4, a significant difference at the level of was found between the pretest and posttest grades of the control group. The arithmetic mean of the pretest of control group was 28.92 and 33.14 for the posttest. This result shows that traditional instruction environment had a positive effect on the academic achievements of the students and hypothesis 2 was refuted.
Hypothesis 3 (there is no significant difference between the pretest and posttest scores of the experimental and control group). In order to determine whether there was a significant difference between the pretest results of experimental and control group, independent group -test was conducted. The results can be seen in Table 5.
As shown in Table 5, there was a slight difference (0.36) between the average scores of experimental and control groups, representing no significant difference. When Table 5 is analysed, it can be seen that the average scores of experimental group () and control group () are very close () and there is no significant difference at a level of () between the groups. According to these results, it can be concluded that there was a homogeneity at the start of the study between the experimental and control group in terms of academic achievement scores.
Hypothesis 4 (there is no significant difference between the posttest scores of experiment and control group). In order to determine whether there is a significant difference between the posttest results of experimental and control group, independent group -test was conducted. The results can be seen in Table 6.
As seen in Table 6, a difference of 3.11 points in the arithmetic averages related to academic achievement of the students was found between the experiment () and control group (). This difference was found significant () at the level of (). The results of the analysis related to the academic achievement scores can be seen in Table 7 and Figure 5.
As seen in Table 7, the experimental group that received instruction with both traditional and Individualized Instruction model had similar scores from the posttest and when the posttests of the experimental group are compared with control group, the difference between them has a significance in favor of the experimental group. In other words, it can be seen from Figure 5 that there is an increase in the achievement of the experimental group.
As shown in Figure 5, pretest results of the experiment and control group are similar; however, there is a significant difference between the posttest results of the experimental and control group in favour of the former.
When considering the achievement of the students in general, the fact that the academic achievement of the experimental group is significantly higher than that of the control group is an important finding of the study proving that the Individualized Instruction Portal is effective in increasing the achievement. The same result was found in the prestudies. One of them conducted by Hsu and Ho [20] did competency-based intelligent mobile learning system. In that study the authors find the system effect to increase the academic achievements. Secondly Butz and colleagues carried out study in 2004 as intelligent web-based education system for the course. This system was designed in the form of static HTML pages online providing assistance to students through course materials, designed to be able to recommend learning goals and coming one after the other into the appropriate sequential web pages. As they stated performed system offers an education that supports individuality [21]. Like we showed in this paper Intelligent Instruction System has a remarkable effect on learning processes. Because of Intelligent Instruction System that supports students, it will be useful as a virtual tutor for students. In Intelligent Instruction System course materials, it will be able to act flexibly and able to respond to the student’s individual needs. In Intelligent Instruction System, issues should never be in the same order by each student. This sequence is generally static in the other system and is identified by those developing systems. Becker et al. (1996) highlight the system as there are “intelligent” features that, according to the personal information collected from students, know how they detect an education and receive the pedagogical decisions [22]. The academic achievement of the experimental group is significantly higher than that of the control group. In this study it is estimated that the most important reason of this increase is that Individualized Instruction Portal improved the motivation of the students in the experimental group and was effective.
4. Conclusion
In this study, it is stressed that education should be done by considering the individual differences of the students. Many studies have been conducted about reshaping learning environments according to students learning styles and their individual characteristics. It was found that creating algorithms for Kolb’s learning style or using mobile systems can be beneficial for selecting courses and taking necessary lessons and also personalized learning progress [23–26].
Similarly Intelligent Instruction System is one of the most important systems that have been developed for the individualized education. Studies show that individualized learning environments meet students expectations with higher rate of satisfaction [1], increasing academic achievements [20], interest, and motivation [23, 27]. In this learning environment it is important to consider the learners’ goals, their former experiences, and their abilities [28] as well as their cognitive styles and competences [29]. For this reason in this research, after exploring the current models of intelligent instruction systems, a new web-based Individualized Instruction Portal (LessonTutor) which pays attention to the differences between the students and presents different learning opportunities depending on the personal needs of the students in learning was developed.
The study group consisted of first grade students in the Department of Software Engineering in the Technology Faculty, Firat University, Turkey.
The data collection process of the study consisted of the development of the achievement test and its implementation. This achievement test was comprised of 47 items and used as the data collection tool. In the validity and reliability analysis of the achievement test, average difficulty was found 0.69 and reliability coefficient 0.75 according to the K-20 formulization; item difficulty indexes values were ranging from 0.33 to 0.90.
The data obtained from the achievement test were analyzed with SPSS. In the analysis section related to the data of academic achievement test, dependent and independent -tests were used where two variables exist.
The implementation of the study was performed in the “Introduction to Computer Science” lecture coded as YMT 113 on the first graders in the Department of Software Engineering of Fırat University in the fall semester. The following conclusions were reached in the study aiming to investigate the effectiveness of designed Individualized Instruction Portal on the students of the experimental and control group:(i)The arithmetic mean of pre- and posttest belonging to experimental group which received instruction in the Individualized Instruction Portal was analyzed statistically with dependent groups -test and 6.97 points of significant difference at a level of () were found in favour of the experimental group.(ii)The arithmetic mean of pre- and posttest of belonging to control group was analyzed statistically with dependent groups -test and 4.22 points of significant difference at a level of () were found in favour of the control group.(iii)The arithmetic averages of pretest belonging to experimental group which received instruction in the Individualized Instruction Portal (LessonTutor) and control group which received instruction in traditional environment were analyzed statistically with independent groups -test and 0.36 points of insignificant difference at a level of () were found between the two groups.(iv)The arithmetic mean of posttest belonging to experimental group which received instruction in the Individualized Instruction Portal (LessonTutor) and control group which received instruction in traditional environment was analyzed statistically with independent groups -test and 3.11 points of insignificant difference at a level of () were found in favour of the experimental group.
In the light of these results, the learning level of the experimental group receiving education with LessonTutor Instruction Portal was higher than that of the control group receiving instruction in traditional instruction environment. It is, therefore, concluded that the Intelligent Instruction System called “LessonTutor” is more effective than traditional Instruction methods.
Competing Interests
The authors declare that they have no competing interests.