Evaluation on the use of animated narrative video in teaching narrative text

In the 21st century, our life is strongly affected by the information technology. Educational technology has been rapidly improved by the development of audiovisual tools. Teachers may choose a number of different types of resources for teaching purposes, including videos and movies. Therefore, this study is aimed at evaluating animated narrative videos from YouTube for the teaching narrative text and identifying potential factors which influence the quality of educational videos. The videos were examined by using assessment rubric to see the quality and suitability of animated narrative videos which might be used in the teaching narrative text. The rubric was adapted from Prince Edward Island (PEI) Department of Education: Evaluation and Selection of Learning Resources. It consists of four criteria, content, structure, instructional design, and technical design In addition, the study presents critical awareness of how these aspects can be interpreted to measure animated narrative videos and at the same time the engagement of the teachers in exploring animated narrative videos used in classroom.


Introduction
Information and communications technologies have significant roles in today's educational case [1]. Nowadays technology cannot be separated from educational activities. Information technology strongly influences the life of the people and plays a significant part in today's human society development. It is necessary to take advantage of the modern technological facilities in improving the development of English language teaching [2]. Studies have shown various advantages to the students related to the general use of technology for EFL, such as the improvement of motivation, self-concept and mastery of the basic skills, the increase of student centered learning and engagement in the learning process [3]. In this case, teachers may use technologies and learning resources for teaching purposes.
Many teachers use various learning resources such as educational YouTube videos.. Founded in 2005, the YouTube has quickly become a leader in online media. It is an internet application through which people can upload, share, and watch. There are millions of messages being presented in this media every day [5]. Educational video is not a specific genre but various genres of video, integrating a variety of production styles, techniques and conventions for the purpose of education [6]. One of the main advantages of using video in the classroom is that the students can both hear and see the language [7].
This study was related to the evaluation of educational YouTube videos. They were very limited in number and most of them were made for non language educational purposes such as medical, patients' health, veterinary and animal sciences [8][9]. The evaluation of educational videos uploaded by Indonesian on YouTube has not been widely exposed. However, the benefit of using YouTube videos in EFL classes has been documented in several studies [10][11][12].
The aims of this study are to evaluate the educational YouTube videos and identify potential factors which influence the quality of educational videos.

Method
This study adopts a mixed method design. Quantitative method was applied in evaluating the quality of the animated narrative videos. Video selection criteria were formulated based on several points. Theoretically, the videos fulfill the classification of educational videos and their duration lasted between 1 to 6 minutes. The videos in this study were produced by Indonesian people. The last criterion was the genre of the narrative text. The videos were selected approximately within a month. The evaluation of the videos was conducted by three experts in EFL and Informational Technology. 13 out of 25 collected videos were classified as educational ones and evaluated by the raters. The instrument used in this study was a video evaluation rubric adapted from PEI of Department of Education: Evaluation and Selection of Learning Resources [4]. They consisted of four aspects, 5 items for content, 4 items for structure, 16 items for instructional design, and 9 items for technical design. The validity of the instrument was administered through content validation. The collected data of the evaluation rubric were analyzed by descriptive statistics using SPSS. This study was also backed up by qualitative data. Since only few studies on video evaluation were conducted particularly in Indonesia, thus qualitative method was applied [13].

Research Findings and Discussion
The research findings and discussion are presented in accordance with the formulated research questions. The first part of this section addresses the evaluation of the animated narrative videos and discusses the aspects of content, structure, instructional design and technical design. The second part discusses the potential factors which influence the quality of the animated narrative videos.

Content
Content is the first aspect and it consists of five items. They were features reflecting the accuracy and appropriateness of the content towards curriculum and students' needs. 2 out of 5 items have averaged rating of 3.0 which indicates that the content of the videos goes with the curriculum and difficulty level of the content. The average mean of the rating was 2.72 which indicate that the content quality of the videos was categorized into "Disagree." It indicates that the content of the videos still needs to be improved particularly for item number 5 in which real world activities are integrated in the content. Figure 1. Raters' rating of animated narrative videos for content aspect

Structure
Structure was the second aspect addressed in the video evaluation. The structure aspect concerns with arrangement of the videos as well as length and its alignment with the planned activities in the videos. The evaluation was made by the raters using 4 items. Only items 3 and 4 have average rating of 3 and the others have average mean 2.72 which indicates that most items of the structure of the videos do not reach the well-structured videos.. This indicates that the structure of the videos was categorized into 'Disagree'.

Instructional Design
The evaluation of the instructional design of the videos covers the examination of its goals, objectives, teaching strategies, and assessment provisions. 16-items were used in rating the instructional design aspect.
As shown in figure 3, the highest average mean was 3 for item number 2. Meanwhile, the lowest average mean was 2.2 for item number 1. The lowest rating score indicated that most of the videos did not clearly state the learning goal and neither adequate assessment. The average mean rating for instructional design was 2.61 categorized into 'Disagree'. The video is well structured (with e.g.… The video is well aligned with the… The video does not exceed 6 minutes.

Technical Design
The last aspect in the evaluation process was technical design which addresses the design of visual and auditory information. Its role in enhancing the learning and developing efficient mental processing is very important. The technical features of this aspect are used as the indicators of good quality videos. As shown in table 4, 3 out of 9 items have averaged mean '3' which indicates that the sound's quality, music and visual effects were appropriate for the topic presented in the videos. The remaining items have average mean less than '3'. The lowest score was 2.6 for item 2. This indicates that improvement on the tone and clarity of the voice should be made. The average mean for technical design was 2.88 which was categorized into Disagree.

Instructional goals and learner…
Resource is suitable for wide range of… Resource promotes student engagement.
Methodology promotes active learning.

Methodology promotes the…
Resource encourages group interaction.
Resource encourages student creativity.
Concepts are clearly introduced.
Concepts are clearly developed.
Concepts are clearly summarized.

Integration across curriculum subjects…
Pedagogy is innovative. Presentation is logical and varied.
Pacing is appropriate.

Items Technical Design
Based on the evaluation made by the raters, various considerations should be made regarding the use of the 13 animated narrative videos in English classroom. The results revealed that none of the aspects under the evaluation reached the average rating 3 (Agree) nor 4 (Strongly Agree). The rating score for content, structure, instructional design and technical design were all under '3'. Each aspect of the evaluation contributes to the overall quality of an educational video. The content of the video must be useful. The video should stimulate, motivate and inform the learner to act on the information. Ideally, learners should consider and incorporate the ideas presented [15]. Another point is the instructional design and structure as they are used to control the design process. Looking at the average rating results of the videos particularly for structure and instructional design aspects which were still below 'Agree' category, it can be inferred that thorough and careful planning was still insufficiently performed. Educational videos should in line with teaching and learning purposes as Clark claims that learning is only influenced by the instructional method, irrespective of which medium is used [16].
With regards to the technical design, the visual and sound quality of the video is an important point as video with poor quality may distract learners' engagement and interest [18]. The results in this study revealed that the narration in the videos was still considered ineffective and inappropriate as it did not correspond to the visuals. In order to overcome this issue, authors of animated narrative video should equip themselves with adequate knowledge on audio and video software and how to use them in order to produce a good educational video as by using appropriate teaching media to present relevant information may increase the efficiency of the self-learning process [19].
Based on the results of the ratings for each video as well as their individual rating descriptions, it was found that the main factors which possibly influence the quality of the animated narrative videos were; lack of preparation, unclear learning goals and objectives, limited knowledge and skills in creating effective and appropriate educational videos. Referring to the identified factors mentioned previously, therefore it is necessary for teachers to carefully select which videos that is countable to be used as learning resource in their teaching. To help them selecting the effective and appropriate videos, Catlin (2013) provides list of points to consider in evaluating the quality of videos. The points are the video has a clear objective, the content is accurate, the creator of the video is an expert or enthusiast on the topic, a balance between educational and entertainment content, the media used in the video should enhance the material presented and the video presents a brief summary in the end.
Teachers could minimize the issues related to the low quality videos by producing their own videos. The following are some considerations to be taken into account while producing videos such as follow three steps production procedure, they are planning, producing and publishing, understand the elements of video design and implementation and master the integrated knowledge or the integrated knowledge is known as Technological Pedagogical Content Knowledge (TPACK).

Conclusion
Available online videos on YouTube or other platforms have high possibilities to be used as teaching aid in the classroom. However, an evaluation regarding its quality is necessary to performed in order to maximize its potential benefits as learning tool for students. In addition, results from video evaluation will improve the quality of the next educational videos which will be produced by experts, teachers or people who are enthusiast on particular fields.