Intelligent Tutoring System with Associative Cellular Neural Network

In self-directed learning such as e-learning, it is significant for learner to recongnize his understanding level. Wrong judgment would provide inefficient learning, and cause a lack of motivation. Test is widely used as an effective method evaluating learner’s understanding level, and we often judge it based only on its score (accuracy rate). For easy system construction, many e-learning systems use a multiple-choice test, which often provides a right answer with no comprehension. Studies on estimating learner’s understanding level by using learning history and test have been reported. One of them is construction of learner model based on accuracy rate and answer time in test (Sobue et al., 2004). However, it requires an enormous amount of data and consideration of learner’s uncertain elements. Another solution is to use heuristicmodel. It is not appropriate to classify learners into at most two groups (either comprehend or not comprehend) due to effective instruction. Then, we add answer time as an objective criterion. Their combination can provide various understanding levels and more precise estimation. The problem that estimates leaner’s understanding level based on objective information can be related to ambiguous classification. Heuristic model is effective for it. Many intelligent tutoring system (ITS) (Gunel, 2010; Nkambou & Bourdeau, 2008) has been developed in recent years. Constructing an accurate learner model is an important element in ITS. The method that generates fuzzy rule and uses classification has been proposed (Takashi et al., 2006). However, it requires enormous data. The methods that use Bayesian Network which utilizes statistical approximation and Support Vector Machine (SVM) which has a high generalization ability have been reported (Okamoto & Kayama, 2008; Sumada et al., 2007). The models have significant problems with computational amount and learning time. This study focuses on Cellular Neural Network (CNN) (Chua & Yang, 1988) which is one of neural network models because it has the following remarkable characteristics:


Introduction
In self-directed learning such as e-learning, it is significant for learner to recongnize his understanding level.Wrong judgment would provide inefficient learning, and cause a lack of motivation.Test is widely used as an effective method evaluating learner's understanding level, and we often judge it based only on its score (accuracy rate).For easy system construction, many e-learning systems use a multiple-choice test, which often provides a right answer with no comprehension.Studies on estimating learner's understanding level by using learning history and test have been reported.One of them is construction of learner model based on accuracy rate and answer time in test (Sobue et al., 2004).However, it requires an enormous amount of data and consideration of learner's uncertain elements.Another solution is to use heuristic model.It is not appropriate to classify learners into at most two groups (either comprehend or not comprehend) due to effective instruction.Then, we add answer time as an objective criterion.Their combination can provide various understanding levels and more precise estimation.The problem that estimates leaner's understanding level based on objective information can be related to ambiguous classification.Heuristic model is effective for it.Many intelligent tutoring system (ITS) (Gunel, 2010;Nkambou & Bourdeau, 2008) has been developed in recent years.Constructing an accurate learner model is an important element in ITS.The method that generates fuzzy rule and uses classification has been proposed (Takashi et al., 2006).However, it requires enormous data.The methods that use Bayesian Network which utilizes statistical approximation and Support Vector Machine (SVM) which has a high generalization ability have been reported (Okamoto & Kayama, 2008;Sumada et al., 2007).The models have significant problems with computational amount and learning time.This study focuses on Cellular Neural Network (CNN) (Chua & Yang, 1988) which is one of neural network models because it has the following remarkable characteristics: 1. CNN can be easily implemented by arranging a simple analog circuit called cell.
2. Calculation efficiency of CNN is better than that of full-connected neural network as represented by Hopfield model because CNN has a local connectivity.
3. Behavior of CNN is expressed by first-order differential equations, and the input-output relation is represented by a linear function with saturated regions.The function provides a high computation accuracy.
In fact, CNN has been widely used in image processing, texture classification (Szianyi & Csapodi, 1998;Yang et al., 2001), time series processing, and so on.In addition, it has been presented that CNN is useful for associative memory (Liu & Michel, 1993).The method is to relate stored patterns to asymptotically stable equilibrium points of dynamics.It enables us that verifying incomplete recall and improving efficiency are easy in pattern classification since information of stored patterns are aggregated in template which represents a connection between each cell and its neighbors.So far, we have presented some associative CNN systems of diagnosing liver diseases, recognizing Chinese characters and detecting abnormal automobiles' sounds, and so on (Namba, 2005;2006;Zhang et al., 2005).This study aims to construct an ITS with associatve CNN which diagnoses learner's understanding level and give appropriate feedback to him.As described above, since associative CNN is useful in classifying ambiguous data, we will apply associative CNN to the diagnosis system.This chapter describes a new diagnosis method with Binary Output CNN(BCNN) (Namba, 2008;Namba2, 2008).As classification information, accuracy rate and answer time are used.The usefulness is verified by using experimental result of Java programming test (15 learners).
The classification results and the validity is discussed.Furthermore, in order to improve versatility of the system, CNN is extended to Tri-valued Output CNN (TCNN) (Namba, 2010;Namba2, 2010;Namba, 2011).By extending CNN function, more kinds of diagnosis information can be used, and representable understanding levels can be increased.Hence, more exact instructions can be realized.Then, a method with associative TCNN is described.In order to evaluate usefulness of the system, associative TCNN estimates understanding levels of 20 learners.The classification results and the validity is also discussed.In addition, in order to evaluate diagnosis ability of associative TCNN, comparison experiment is performed with linear function which is conventional estimation method.

Cellular Neural Network
CNN is composed of simple analog circuits called cell.Each cell is connected with r−neighbors, which means a local connectivity.Fig. 1 shows 7 × 7a n dr = 2 CNN.In this figure, the black cell is C (4,5), and gray cells are the neighborhood.In r−neighborhood m × n CNN, the i-th row, j-th ,where x ij , y ij , u ij and I ij show a state, an output, input variable, and a threshold, respectively.T ij and S ij are connection coefficients which show influence from the neighbor cells.* means the following opeation: For example, the dynamics of cell C(4,5) in r = 2,7 × 7 CNN is represented as the following: : neighbor cells of CNN used in this study is u ij = 0 for simplicity.Hence, the dynamics is Moreover, y ij is a function of x ij as the following: Fig. 2 shows the binary output function.Eq.( 5) guarantees a high computation accuracy.By using vector notation, the differential equation of m × n CNN can be represented as ,where x, y and I show a state, an output, and a threshold vector, respectively.T is a template matrix.

Design of associative BCNN
Associative BCNN can store a lot of patterns because they are related to asymptotically stable equilibrium points of dynamics.We prepare p state vectors . In other words, we can obtain Eq.( 7).
It is evident that α i , β i , T and I concurrently satisfy the following equations: Let matrices A and B be We are able to obtain the following equations: T and I which satisfy these expressions must exist in order for the CNN to have α i as stored vectors.We are easily able to find T and I by adopting the following method.
If we focus on the computing at k−th cell in CNN(k = n(i − 1)+j), its conditional equation is given by ,where b k and t k are the k−th row vector of matrix B and T .A large number of null elements are included in the vector t k .Using the property of the r−neighborhood, we obtain Eq.( 12) by 126 E-Learning -Organizational Infrastructure and Tools for Specific Areas ,where A r is a matrix having removed the elements not belonging to r−neighborhood of the k−th cell from matrix A. b k and t k are the similar meaning vectors.The amount of computing can be decreased by these procedures.Generally, the matrix A r is not a square matrix.Hence, we solve t r k by using a singular value decomposition as the following: Hence we have This solution is the minimal norm of Eq.( 12).
[λ] 1/2 is a diagonally dominant matrix consisting of square root of the eigenvalue of matrix [A r ] T A r .U k and V k are the orthogonal matrices, respectively.
In the CNN designed by the method described above, stored patterns theoretically correspond to each equilibrium point of dynamics.CNN can recall a pattern by solving Eq.( 6) when an initial state is given.In other words, it will converge on the stable equilibrium point of the optimal solution of differential equations.

Overview
Fig. 3 shows the overview of associative CNN diagnosis system.In advance, we define understanding levels, and correspond them to stored patterns in associative CNN.Learner pattern is generated from his test data.After that, a stored pattern can be obtained by the self-recall process of associative CNN.The understanding level which corresponds to the pattern is the diagnosis result.Due to the characterisic of associative CNN, stored patterns are not always recalled.This means that the system can't diagnose his understanding level.

Expressing cell and answer pattern
We here relate understanding levels to stored patterns of associative CNN.In order to design the diagnosis system, we use prior knowledge as the following: • High understanding level has high accuracy.
• Higher understanding level has high accuracy and short time.
• Low accuracy and short answer time may mean that he makes a careless mistake.
They can be expressed by combination of cells.We allocate information of a question to two cells which means answer and time.Each cell denotes the following: 1. Answer cell means that an answer is right/wrong (Discrete).
2. Time cell means that an answer time is short/long (Continuous).
Table 2 shows understanding levels defined per question.Moreover, by aggregating them to the number of questions, answer pattern can be generated.Fig. 4 shows the expression method.4. Expression of answer pattern by associative CNN

Stored patterns
This study defines three understanding levels, and design four stored patterns corrresponding to them.Fig. 5 shows them, and Labels A-D correspond to those of Table 2.

Generating input pattern
The answer time is transformed into −1 ∼ +1 for input to associative CNN (scaling).Hence, in order to transform an actual answer time d k into normalized one q k in the k−th question, Table 3. Question level defined in the experiment we use ,where σ k and tk are standard deviation and average of all learners in the k−th question.

Experimental results by associative BCNN
15 learners who studied Java programming were experimented.They were randomly divided into two groups (ten and five students), and experimented in each group.The former belongs to Group X, and the latter belongs to Group Y.

Experiment 1 -Group X
The learners in Group X were experimented.At first, all 50 questions were divided into three levels according to accuracy rate of Group X.As shown in Table 3, question level was defined, and all questions were classified.Moreover, the average and standard deviation of answer time in Group X were obtained, and all data were transformed by using Eq.( 15).The diagnosis results are shown in Table 4. "Others" in   We next made a similar experiment in Group Y.The same value as tk and σ k of Group X were used.Experimental results are shown in Table 5.In this experiment, diagnosis rate was 100 %.
Associative CNN has a classification function of ambiguous data, and doesn't require a great number of data such as a fuzzy rule.Hence, it can be an useful model for such a problem.Moreover, a system should immediately provide learners with feedback in real-time environment such as WBT (Web Based Training) system.As Experiment 2, if the system could classify without recalculating tk and σ k , its usefulness would be much higher.

Experiment 3 -Comparison experiment
In this section, multi-layered perceptron (MLP) which is one of representative neural network model was applied to this diagnosis problem in order to compare with associative CNN performance.The typical three-layered perceptron with Back Propagation Algorithm was used.Fig. 6 shows MLP model used in the experiment.Training data were four stored patterns as shown in Fig. 5. Learning was iterated until the mean square error between training data and the output was less than 0.001.When a value of output unit was more than 0.75, the pattern corresponding to it was the estimation result.Tables 6 (a) and (b) show the results.Most of patterns were classified as "Others", in other words, "not diagnosed".To compare with the diagnosis result by associative CNN (Tables 4 and 5), it is evident that diagnosis rate of associative CNN is superior to that of MLP in both cases.Associative BCNN is effective to diagnose learner's understanding level because it has a high diagnosis capability.However, it has a significant problem with lack of representable understanding levels.Though we can basically overcome it by increasing cells, and computational amount is also be increased.

Extended associative CNN -TCNN
As described above, Associative BCNN is effective for diagnosis of understanding level.However, it has a difficulty of few understanding levels.In order to overcome it, extended Associative CNN is designed, and more versatile system is constructed.

Tri-valued output associative CNN
The principle of Tri-valued output CNN (TCNN) is similar with that of BCNN, and we can easily realize by redesigning output function as The parameter s provides a new saturated region, and L enables us to adjust saturated regions.If s = 0andL = 0, the function is conventional BCNN.Fig. 7 shows the output function.Design method of associative TCNN is also similar with that of associative BCNN.All you have to do is to change each element of α i in Eq.( 7) is −1, 0, +1. .Expression of answer pattern by Associative TCNN

Design of associative TCNN diagnosis module
By extending associative CNN to tri-valued output, the number of representable information for diagnosis can be increased.In other words, more prior knowledge can be realized in cell expression.Then, question level (difficulty) is newly added.That is, 1. Answer cell means that an answer is right/wrong (Discrete).
2. Time cell means that an answer time is short/long (Continuous).
3. Level cell means that a question is easy/standard/difficult (Discrete).
Fig. 8 shows examples of answer pattern per question.(a) means right/short/easy, and (b) indicates wrong/medium/difficult.Moreover, we can make a total answer pattern by arranging all answer patterns as shown in Fig. 9 as is the case with associative BCNN.We next generate stored patterns which correspond to understanding levels.This study defines six grades (four understanding levels) in a question as shown in f f Table 8.Understanding levels understanding levels in accordance with combination of label and level as shown in Table 8, and design stored patterns which correspond to them.

Evaluation experiments and results
The purpose of experiments is to evaluate how exactly associative TCNN can diagnose learner's understanding level.Experiment condition is described as follows:

Comparison experiment
In order to evaluate the accuracy of our system, we make a comparison experiment.We use a linear function according to the related work (Sumada et al., 2007) as the following: ,where L t , L a are answer time and accuracy, respectively.U is understanding level.This equation is also based on our prior knowledge.It is appropriate as a comparison model with ours.
Comparison experiment is performed as the following: 1.All 38 questions are divided into two groups so that their difficulties will be the same.One is estimation part, and the other is evaluation part.

2
. Accuracy and answer time are obtained from estimation part in order to be used in estimating understanding level.

3
. Accuracy and answer time are obtained from evaluation part in order to be used in evaluating estimation results.
4. True understanding level is obtained from 3.
5. Precision of associative TCNN and that of linear function are considered by comparing with true understanding level obtained in 4.
In case of estimation by linear funcion, k is changed from 0.1 to 0.9, and the best is adopted as the estimation result.

Conclusion
The early half of chapter described a new diagnosis system of learner's understanding level by using associative Binary Cellular Neural Network (BCNN).The results obtained are as follows: 1.At first, ten learners (Group X) were divided all questions into three levels (difficulties) in accordance with accuracy rate.Their understanding levels could be appropriately diagnosed by the system in each difficulty.As a result, diagnosis rate was about 93.3%.
It implies that the result can be appropriate according to the relation between question level and result of understanding level.
2. Moreover, we made a similar experiment on five learners (Group Y).Consequently, diagnosis rate was 100%.It implies that associative BCNN is useful as an diagnosis module in terms of versatility.
3. The similar diagnosis experiments were performed by Multi-Layered Perceptron (MLP).Consequently, the diagnosis performance of associative BCNN is better than that of MLP.
BCNN has an important problem with lack of representable understanding levels.Then, the latter half of chapter described an improved diagnosis system by using associative Tri-valued Cellular Neural Network (TCNN).The results obtained are as follows: 1. We first defined five understanding levels, and designed associative TCNN according to them.
2. We next made a diagnosis experiment with associative TCNN.Participants were 20 learners who studied "Probabilities".Consequently, associative TCNN could completely diagnose all of them into expected understanding levels.It was recognized that the diagnosis results were valid in accordance with accuracy and answer time.
3. In order to compare our system with conventional method, we also made a diagnosis experiment by linear function.From comparison of precision, we confirmed that associative TCNN which comprehensively diagnoses was better than linear function which partly estimates.
The diagnosis by associative TCNN plays a role in rough-classification in pattern recognition.Standard associative TCNN is used in the experiment.Hence, optimized associative TCNN will probably provide better results.Our future work is to improve diagnosis performance with optimized associative TCNN.

Acknowledgment
Part of this research was supported by a Grant-in-Aid for Scientific Research from the Ministry of Education, Culture, Sports, Science and Technology (21700829), Japan.

Table 1 .
The matrix t 45

Table 2 .
Understanding levels defined per question

Table 6 .
Diagnosis Result by MLP 4.2 Experiment 2 -Group Y

Table 9
shows diagnosis results.All values in the table are average.This result shows that associative TCNN could completely diagnose all learners.We next evaluate the validity of diagnosis result.For instance, learners in the cluster "A" should have higher understanding level in all levels.It has high accuracy and short time compared with averages of all in any level.Hence, the diagnosis result of cluster "A" is valid.Learners in cluster "C" should have high or higher understanding level in Standard and Easy (Level II and I), and low in Difficult (Level III).Learners in cluster "D" should have high or higher understanding level in Easy, and low in Difficult and Standard.Learners in cluster "E" should have low understanding level in all levels.We can similarly conclude that the diagnosis results of clusters C,D, and E are valid.
Table10shows a comparison of precision.The precision of associative TCNN is superior to that of linear function.These results represent that synthetical diagnosis of associative TCNN is better than partial estimation of linear function.