Development of Educational System with the Android Robot SAYA and Evaluation

This chapter describes an educational system with a tele-operated android robot, named SAYA, that can express human-like facial expressions and perform some communicative functions with its head and eye movements, and it is utilized as a role of a teacher. Two kinds of field experiments were conducted to investigate effectiveness of this educational system in actual educational fields. An experiment was conducted at both an elementary school and a university to estimate age-dependent differences of its effectiveness. The other experiment was carried out to verify whether children's interest, motivation, and concentration to the class, and science and technologies were enhanced.


Introduction
For the last decade, a wide variety of robots have been developed and studied, which can behave effectively and offer many kinds of services in our daily lives through interaction with human. In the near future, such robots are expected to be able to offer not only physical assists but also informative and emotional supports. Toward this purpose, it is very interesting to investigate what kinds of functions, mechanisms, and intelligence are required for such robots and to investigate interaction manners between human and robot in daily lives. Therefore, many kinds of robots, called "communication robot", have been developed and applied to not only laboratories but also our daily lives (Bauer et al., 2009;Burgard et al., 1998;Fujita, 2001;Hayashi et al., 2007;Kanda et al., 2004;Shiomi et al., 2007;Siegwart et al., 2003;Wada et al., 2002). For example, the pet-type robot, named AIBO (Fujita, 2001), was commercialized around ten years ago, which is one successful example of the robots which can behave in our living space. Also, a seal type robot was developed as a mental therapy robot and its effectiveness for elderly people was verified through the field experiments in nursing-care facilities (Wada et al., 2002). Such kinds of animal-type robots interact with human emotionally by performing endearing behaviors. On the other hand, humanoid-type robots have been developed, which have human-like body such as a head and arms to express more human-like behaviors. In researches of humanoid-type robot, body movements such as gestures, nodding, eye-direction, and facial expressions are effectively utilized as non-verbal behaviors to interact with human naturally (Breazeal & Scassellati, 1999;Bremner et al., 2009;Imai et al, 2001; Kamasima et al., 2004;Watanabe, et al., 1999). For example, Robovie (Hayashi et al., 2007;Shiomi et al., 2007) was actually used in real-environment such as a train station or a museum, and it interacted with humans and offered information of facilities by utilizing its human-like behaviors. Furthermore, communication robots are also used in educational fields (Han et al., 2005(Han et al., , 2009; Kanda et al., 2004;Tanaka & Kimura, 2009), and they can teach students and learn with students through interactions. A merit of educational applications of communication robots might be to encourage children to be interested in science and technology.
In addition to the robots with mechanical looks, androidtype communication robots with highly human-like looks have been developed (Ishiguro, 2005;Oh, et al., 2006). A merit of android robots is that they give people a feeling of human-like presence as if people were interacting with a real human. Therefore, if android robots were used as an interface of communication systems and interacted with humans using human-like behaviours, people could interact with robots using same manners as in interaction with real humans. Actually, for example, the effectiveness of the tele-operated android robot was verified on conveying presence of a human who was in different place rather than existing media such as a speaker or a video-conference system .
In this chapter, a remote class system is introduced as an applications of android robots, where the android robot SAYA (Hashimoto, 2005) (Fig. 1) is used as an interface. Here, in particular, the investigation of its effectiveness for elementary school children is interesting because children tended to be interested in learning with a robot and they were motivated to learn a foreign language as shown in the previous research . Hence, the proposed educational system with the android robot SAYA is also expected to contribute to children's motivation to learn. In this study, two kinds of field experiments were conducted to investigate the effectiveness of the proposed educational system in actual elementary schools. One of them was conducted for both children (elementary school students) and adults (university students) to estimate age-dependent differences of its effectiveness. The other one was carried out to verify whether there are significant changes in children's interest, motivation, and concentration to science classes and technologies between before and after the class which was conducted by the proposed educational system.
The structure of this chapter is as follows. In Section 2, the android robot SAYA and its communicative functions are introduced. Section 3 describes the system structure of the remote class system with the android robot SAYA. Section 4 describes the detailed experimental conditions and procedures of two kinds of field trials which are conducted in actual educational fields, and their results and the contributions of this research are represented and discussed.the contributions of this research. This chapter is concluded in section 5. Fig. 1 (a) shows the android robot, named SAYA. It has anthropomorphic appearance and one of its characteristics is to express human-like facial expressions (Fig. 2). The main part of SAYA is the face part which is called "Face Robot" and implemented to a mannequin body. The following is the detailed description of the Face Robot.

The structure of the Face Robot
Toward the achievement of a humanoid robot with anthropomorphic properties making the robot so real that it cannot be distinguished from a living human, the Face Robot have been developed (Kobayashi & Hara, 1993;Hashimoto et al., 2006), and Fig.1 (b)   In the eye movements, it has 2 DOFs that include both yaw rotation and pith rotation, and both eye-balls move together because both eye-balls are linked to each other, and these two eye-balls are driven by two DC motors. A small CCD camera is embedded in an eye ball for image processing. For example, the Face Robot is able to recognize human face by extracting skin color and it can track a human.
The mechanism for the head movements consists of the head part and the neck part in which a coil spring is utilized, and the head movements are achieved by combining the head rotations and the neck flexion. Here, the neck part is able to bend flexibly by benefiting from the coil spring to mimic flexible neck movements of human. As a result, the Face Robot has 4 DOFs in the head movements as shown in Fig.  3(a); 2 DOFs for the neck part and 2 DOFs for the head part.
The lateral flexion of the head is achieved by only the neck flexion ("Roll"), and the forward and the backward flexion are achieved by combining the head pitch-rotation ("Pitch1") and the neck flexion ("Pitch2"). The horizontal head shaking is achieved by only the head yaw-rotation ("Yaw").
McKibben pneumatic actuator is adopted to control the facial expressions and the head movements. One of its characteristics is the ability to generate too large force for its relatively small size and light weight, and it can be distributed to curved surface like the skull of the Face Robot because of its flexibility. Fig. 3(b) shows the actuator layout.
In the face part, one or two actuators are used in each CP, and the neck part is driven by 4 pairs of antagonistic actuators.

Methodology for generating facial expressions with the Face Robot
Facial expressions are able to express individual emotions significantly and play an important role in face-to-face communication of humans as a non-verbal media (Mehrabian, 1968), and facial expressions seem to contribute to achieve natural communication between humans and robots. Therefore, generating natural facial expressions similar to human is required for robots to interact with human naturally and emotionally. Almost all related studies of generating facial expressions adopt "Action Unit (AU)" approach, and AUs were defined in Facial Action Coding System (FACS) proposed by P. Ekman et al. (Ekman & Friesen, 1978). AUs express motions of mimic muscles as 44 kinds of basic motions, and 14 AUs which are shown in Table 1 are required to generate 6 typical facial expressions; "Anger", "Disgust", "Fear", "Happiness", "Sadness", and "Surprise". Referring to this approach, 19 control points were modelled, and their movable directions are shown in Fig.  1(c). The combinations of CPs are also defined to achieve each AU, and various facial expressions can be realized with the face robot by combining movements of some CPs. Fig. 2 shows the examples of 6 typical facial expressions of the Face Robot, and high correct recognition rate of every facial expression was achieved in the previous study (Hashimoto et al., 2006). In addition to the succeeding in the development of the typical face robot, the specialized face robot have been developed, which can remarkably mimic an actual living-human and has highly realistic presence (Hashimoto et al., 2008).  Figure 4. System configuration of remote class system with the android robot SAYA 3. Educational system with the android robot SAYA An educational system, particularly a remote class system, has been developed as a practical application of the android robot SAYA. In considering practical aspect of communication robot, to achieve smooth and natural communication between human and robot is one of the most important problems. However, intelligence technologies of robots are generally still lacked to interact with human and act in daily lives autonomously even though variety of autonomous robots have been developed and studied so far. Autonomous communication robots are currently simply able to interact with human in well-designed interaction scenarios and in well-defined environment as well. Meanwhile, tele-operated robot which is manoeuvred by a hidden operator has the advantage in terms of practicality because it seems to conduct behaviors and interactions autonomously from the viewpoint of a human who interacts with the robot even though the robot is controlled by tele-operation. Particularly, if an android robot is used as an interface of tele-operated communication system, it will give people a strong feeling of presence and make people feel like they are interacting with real human as described in Section 1 . In addition, it is expected that elementary school students are very interested in interaction with an android robot and they actively participate in the class which is conducted by the android robot. The detailed configuration of the proposed educational system is described as follows. Fig. 4 shows the system configuration of the proposed educational system, and the android robot SAYA is utilized as the role of a teacher.

System configuration
In the classroom, there are SAYA and some control equipment, and the control system of SAYA requires a compressor and an electro-pneumatic regulator to control contractions of McKibben artificial muscles. In addition, the electric-pneumatic regulator is controlled by the control computer ("Control PC") to control SAYA's facial expressions and head movements, and the control computer also controls SAYA's eye-direction and utterances as well. A microphone and a video camera are used to obtain visual and sound information of the classroom.
In the operation room, there are two monitors. One of them is used for the control, and the other one is used for the observation. The operator is able to monitor students' behaviors through the observation monitor, and he can manoeuvre SAYA's utterances and actions by sending commands from the operation PC ("Operation PC") to the control PC ("Control PC") through the LAN. The control PC executes robot's utterances and actions based on received commands. Specifically, captured images from SAYA's CCD camera and the video camera are transmitted to the operation PC, so the operator can observe the classroom from the two viewpoints. As a result, the operator can move SAYA's viewpoint by controlling its eye and head directions corresponding to these visual information, and SAYA is able to look around the classroom and look at a student. The operator is able to hear students from the speakers and respond to students as well.

Interactive behaviors
In the developed system, there are two operation modes which include "lecture mode" and "interaction mode".
In "lecture mode", SAYA gives some explanations about some contents of a class to students while looking around the classroom by the tele-operation, and some slides which are projected on the screen in front of the classroom to help students to understand simultaneously. SAYA's utterances are previously prepared along a scenario of a class.
In "interaction mode", SAYA performs interactive behaviors such as looking around the room, paying attention to a student, and talking to a student. In order to talk to students, SAYA is able to respond with registered short sentences such as "Do your best!", "Be quiet!", "Don't look away" and so on. If students' questions are beyond SAYA's default database, SAYA replies with a word of kindred meaning which is selected by the operator. In addition, SAYA is able to express facial expressions according to its utterance. For example, when SAYA says "Be quiet!", it executes the facial expression "anger". Also, SAYA is able to call students' name individually, because the names of the students who participate in the class are recorded in advance. Here, female voice that was recorded beforehand are used as SAYA's voice.
The operator can execute these interactive behaviors easily by using a simple operation interface. In addition, SAYA apparently seems to conduct classes and interactions with students autonomously because SAYA is performed by the tele-operation.   In part (b), there are short sentences and students' names that are pre-registered for brief interaction with students. In part (c) and (d), there are some icons that correspond to robot's behaviors such as the facial expressions, the eye and head movements. By clicking these icons, the operator is able to execute such behaviors.

Operation interface
As shown in Fig. 5(b), the operator can observe a classroom and hear students' utterances through the display and speakers, and he is able to click icons easily by using a touch panel or a mouse.

Field trials at educational fields
In order to evaluate the proposed educational system, two kinds of experiments were conducted in actual educational fields, particularly in elementary schools. The detailed procedures and the results of each experiment are described in 4.1 and 4.2.

Field trial for estimating age-dependent differences of effectiveness
As a first step, the field trials were carried out on both elementary school students and university students to estimate the age-dependent differences of the effectiveness, and we investigated whether students' interest, motivation, and concentration to the proposed educational system differed according to their age or not. Such differences were estimated by comparing the questionnaire results between elementary school students and university students.

Experimental setup a) Content of the class
In the first experiment, "robot class" was conducted as a science class in which SAYA introduces itself and other interesting robots. Table 2 shows the flow of "robot class". First of all, in Scene 1, SAYA gives the self-introduction to students and begins the class. In Scene 2, SAYA interacts and talks with students briefly, and then SAYA asks students, "What kinds of robots do you know?" or "What kinds of robots do you want?". Scene 3 is the introduction of the robots which can demonstrate high performance in hazardous environments such as in disaster site, and so on. Scene 4 is the introduction of the robots which can assist human physically in medical facilities and living environments. Scene 5 is the introductions of the robots which are expected to demonstrate high performance in the near future and offer services while interacting with human in our daily lives.   In Scene 6, SAYA summarizes its talk and closes the class. In addition, an operator sometimes executes SAYA's interactive behaviors such as looking at a student or talking to a student during each scene, and the class takes around 30 minutes.

b) Participants
The experiments were conducted 4 times in two elementary schools as shown in Table 3. As a result, 38 elementary school students who belong to from 1st to 6th grader participated.
In addition, the same experiment was conducted for 30 university students who were in their twenties to evaluate the age-dependent differences of effectiveness by comparing with the elementary school students. In the experiment, they were divided to three groups and each of the three groups participated in the class separately.  Fig. 6 shows an experimental environment, and a standard classroom was used. As described in Section 3, the android robot SAYA was put in front of the classroom, and a screen was placed next to SAYA to show slideshows that were used to explain and help the students to understand the content of the class. An observation camera and a microphone were placed in back of the classroom. The sitting positions of the students were set within both the visual angles of SAYA and the observation camera, and 4 or 5 students sat at each desk.  were also asked to comment on the class and the proposed system.

e) Experimental procedure
Before the experiment, an experimenter let the students sit at the assigned positions because an operator needed to identify the students so as to look at a student or call students' name individually. Then, the experimenter gave explanations about the experiment to the students. After that, the experimenter exited the classroom, and the operator controlled SAYA to conduct a greeting and to begin the class. During the class, the operator sometimes talked to the students, and he also asked some questions or gave cautions and advices to the students through SAYA. After the class, the students were asked to answer the questionnaire. Fig. 7 shows the questionnaire result, and the average and the standard deviation of each question are also shown in the figure. As a result, the evaluation values except Q. 3 of the elementary school students are higher than those of the university students. The results of Mann-Whitney's U-test are also shown in Fig. 7. The significant differences at p < .01 and p < .05 between the evaluation values of the elementary school students and the university students in Q. 2, Q. 6, Q. 7, Q. 8, and Q. 9 are proved. It is found that the elementary school students are easy to accept the proposed educational system with the android robot SAYA more than the university students because the elementary school students estimated significantly high in following questions, Q. 7 "Did you feel familiarity with SAYA?", Q. 8 "Did you feel that you are being watched?", and Q. 9 "Did you feel eeriness in SAYA?". Also, the elementary school students actively participated in the class more than the university students because there are significant differences in Q. 2 "Did you have a good time in the class?" and Q. 6 "Do you want to participate again?".  The comments from both the elementary school students and the university students are shown separately in Table  4. Both of them indicate the lack of movements of SAYA (e5, u2) , and the university students particularly indicated the unnaturalness of SAYA's movements (u3).

Experimental results and discussions
Thus, the improvements in its movements are required. SAYA also needs not only the ability to conduct the conversation along a scenario but also the ability to talk freely, because there is the comment of u10 "I would like to talk to SAYA freely". On the other hand, it is confirmed that calling name is an effective interactive behaviour because they were surprised that SAYA can call their name individually (e8, u7), and the advantage of the remote control is also confirmed because they were surprised at the conversation ability of SAYA (e9, u4). The elementary school students indicated their interests in the class such as e10 "The class was fun" and e11 "I would like to participate again" while the university students mainly emphasized SAYA's abilities and functions. Therefore, it is proved that the elementary school students were more interested in the class with SAYA than the university students.
It is thought that the elementary school students are involved in high novelties such as robots, particularly android robots, because they have less opportunity to see or touch robots. As a result, the effectiveness of the proposed system is confirmed in educational fields, particularly to younger age brackets such as elementary school students.

Experimental setup
As a second step, another field trial was also conducted at an elementary school to verify the effectiveness of the proposed educational system in an actual science class. Students' interest and motivation to the class were also estimated by using a questionnaire.

a) Content of the class
In the experiment, "the principle of leverage" was adopted as a topic of a usual science class of elementary school. This topic is generally difficult for children because it contains both mathematical elements and experimental validations. In order to prepare teaching materials of the class, the science textbooks which are commonly used in elementary schools were referred. "The principle of leverage" represents mechanical properties of leverage. The lever is a rigid object which is used either to amplify small force to move larger force (load), or to change small distance and speed of the end of the lever to larger distance and speed of the opposite end. That is, it is a good example of the principle of the moment. The leverage is one of the simple machines, and is also a usual topic of science in elementary school. Table 5 shows the flow of the science class about "the principle of leverage". First of all, in Scene 1, SAYA gives the self-introduction to students like "robot class", and the class is begun. Main topics of the class consist of the three scenes (Scene 2-4). In Scene 2, SAYA gives the explanations about the theory and mechanical advantages of leverage with some slides. In this scene, three important points (i.e., a pivot point, a point of effort, and a point of load) of leverage are explained. SAYA then shows some familiar examples of leverage that are seen in our daily lives in Scene 3. For example, scissors, bottleopeners, tweezers, and so on. After that, in Scene 4, SAYA lets students experiment to confirm the principle of leverage with an experimental kit, and they are able to experience the balancing theory of a lever. In Scene 5, SAYA summarizes its talk and closes the class. The class takes around 30 minutes like the experiment of "robot class".

c) Experimental environment
A standard classroom was used like the experiment in "robot class", and the android robot SAYA was put in front of the classroom as the role of a teacher. In this experiment, a plasma display was used to show slideshows, and an experimental lever kit was put in each desk as shown in Fig. 8(a)(b). Four students per desk sat down at the assigned positions.

d) Evaluation method
A questionnaire was organized in order to investigate how students' interest and motivation were affected through the experiment, and the questionnaire investigations were conducted before and after the class. Table 6 shows the questionnaire that consists of 12 questions (Q. 1 -Q. 12). Here, the first 4 questions (Q. 1 -Q. 4) were used for the investigation before the class as a brief questionnaire, and the rest of the questions (Q. 5 -Q. 12) were used after the class. Except that Q. 12 was evaluated with the "yes" or "no", each question was evaluated on a scale of -2 to 2, where 2 is the most positive evaluation. Q. 1 and Q. 5 (interest in science and technology), Q. 2 and Q. 6 (interest in science classes), Q. 3 and Q. 7 (interest in robots) , Q. 4 and Q. 8 (interest in a class conducted by robots) are paired to investigate the contrast of students' interests and motivation between before and after the experiment.

e) Experimental procedure
The procedure of the experiment was almost the same as the experiment of "robot class" described in 4.1. At first, an experimenter let the students sit down at the assigned positions. He then explained the flow of the experiment briefly and asked the students to answer the brief questionnaire which consists of 4 questions (Q. 1 -Q. 4).
After that, he exited the room and an operator began the science class by controlling SAYA. The science class was conducted along the scenario described in Table 5.
During the class, the operator sometimes interacted with the students just like the experiment of "robot class". After the class, the experimenter asked the students to answer the questionnaire which consists of 8 questions (Q. 5 -Q. 12). Table 6 also shows the averages (ave.) from Q. 1 to Q. 11, and the numbers described in parentheses are the standard deviations (s.d.). The numbers of students who answered "yes" or "no" in Q. 12 are also shown. In addition, Fig. 9 shows the comparisons between Q. 1 and Q. 5, Q. 2 and Q. 6, Q. 3 and Q. 7, Q. 4 and Q. 8. Table 6 shows that students' interests and motivation were affected through this experiment. That is, the evaluation values of Q. 1, Q. 2, Q. 3, and Q. 4 are higher than the values of Q. 5, Q. 6, Q. 7, and Q. 8 respectively. Wilcoxon signed-rank test were also applied to each pair,and the results are shown in Fig. 9. The results reveal a significant difference in the pair of Q. 2 and Q.6 (p < .01 ). Therefore, the class conducted by SAYA enhanced the students' interests and motivation to science classes. In addition, it is found that the concentration of the students was low because the evaluation value of Q. 9 (concentration on the class) is relatively lower than that of other questions. The results of Q. 10 and Q. 11 indicate that the students can easily understand SAYA's explanations and teaching materials, and the students want to participate in the class again. In Q. 12, 17 out of 22 students answered "Yes". That is, it is confirmed that the operator replied to them correctly through SAYA. Fig. 10 shows some photos of the experiment at the elementary school. Fig. 10(a) shows the scene in which SAYA gave the explanations about the leverage along the scenario, with the students concentrating on hearing SAYA's talk and paying attention to the screen. SAYA sometimes looked at a student and asked some questions as shown in Fig. 10(b). Fig. 10(c) shows the scene in which the students raised their hands and attempted to answer SAYA's question. In the class, the students also experimented for discovering and confirming the principle of leverage with an experimental lever kit as shown in Fig. 10

Conclusion
In this chapter, the remote class system is proposed, where the android robot SAYA is used as a teacher. SAYA has highly anthropomorphic appearance, and the remote control system of SAYA was developed for the proposed remote class system. The developed system allows an operator to easily control SAYA's behaviors such as the facial expressions, head movements, eyedirection, and utterances, and the operator is also able to observe students' behaviors remotely.
Two kinds of field trials were conducted in actual educational fields to investigate the effectiveness of the proposed educational system. One of them was carried out for both elementary school students and university students to estimate the age-dependent difference of effectiveness, and "robot class" was conducted as the topic of a science class. The other field trial was conducted to verify its effectiveness in actual science class. "The principle of leverage" was adopted as the topic of usual science class, and the students' interest and motivation to the class were estimated.
From the experimental results, the followings are confirmed in terms of the positive effects and the possibility of the proposed educational system in actual educational fields, especially in elementary schools.
 The elementary school students are easier to accept the proposed educational system and more actively participate in the class than the university students.  The proposed educational system enhances the elementary school students' motivations to science classes.
Our future works are to conduct long-term experiments at elementary schools and evaluate its educational effects on children, and the proposed educational system also should be compared with other existing remote communication system such as tele-conference system to evaluate its advantages. Also the contrasts between the proposed educational system and human teachers should be investigated.