1 Introduction

The outbreak of COVID-19 and advanced information and communication technologies have changed people’s learning behaviors. The World Economic Forum has reported that technology can extend knowledge outside the classroom and provide learners with more learning opportunities. In addition, learners can customize their learning model according to their needs [73]. Nowadays, e-learning has become an indispensable existence in the teaching field. In order to attract more attention, many knowledge transmitters use “videos” or “multimedia” as a medium for transmitting information and knowledge. Augmented Reality (AR) technology also has been used to increase users’ learning interests [59, 68]. Due to COVID-19 epidemic, the use of multimedia, such as instructional animations, videos, and virtual reality (VR) in distance teaching, has attracted lots of academic attention [3, 6, 22, 56, 63]. Compared with text, “video” has a dynamic visual effect. During the fast browsing process, it will attract the viewer’s attention and help the viewer to receive the content delivered in a short time [18].

In the teaching videos, the elements to be conveyed through visualization are generally called “design elements,” which can include text, images, animation, sound, etc. Using “visual cueing” which can promote information through visual effects is more prominent to attract attention. The cognitive theory of multimedia learning also mentioned that appropriate use of “cues” can reduce cognitive loads and make it easier for learners to understand the content delivered [44]. In “emotional design”, “emotions” are the individual’s judgments about the world. These judgments refer to the reactions and interactions that the individual produces when stimulated [15, 21, 38, 51, 53,54,55, 70]. In 2004, Norman [51] presented the theory of emotional design, thinking that how to stimulate positive and negative emotions when interacting with the design is very important.

When people use instructional videos for learning, they all hope to acquire knowledge quickly and clearly [6]. Therefore, instructional designers play an important role between learners and textbooks, and each design element must be carefully used to make it can be used to its maximum effect in teaching videos [72], combining learning objectives with multimedia, achieving interaction, and delivering teaching content to learners more effectively and accurately. At present, many researches on design elements only discuss learning effectiveness or satisfaction. When the design element has learning effectiveness or is highly satisfied, the textbook designer is recommended to use the design element [7, 28, 29, 58, 71]. However, the results analyzed only through learning effectiveness and satisfaction are not accurate enough. In addition, designers often fail to consider the perspectives of learners when designing elements, but the opinions of these users are often the key to the success of teaching videos.

To discover the expectations and needs of users for digital instructional videos, this study defines “visual cues” [7, 28, 29] and “emotional design” [58, 71], containing eight elements to analyze learners’ needs of categorization of each element, and discover learners’ expectations for teaching videos. These eight elements are arrow cueing, dynamic arrow cueing, spreading-color cueing, contrary to cueing, font style, color application, anthropomorphic, and audiovisual complementarity. Refined Kano Model [74] has been employed to categorize elements and provide suggestions based on the results to help designers make instructional videos.

Moreover, this study also conducts in-depth data analysis through decision trees algorithm, and stratification analyses using different variables (such as design expertise, using frequency, and usage experience, etc.) to find out the key design factors that affect learners’ learning. Depending on the learner’s background, the use of e-learning experience, using frequency, and the length of the learning video, our results could provide for reference when designing teaching videos. Instructional designers can better understand how to effectively use design elements, so that the teaching videos can achieve the best learning effect.

The current paper has been structured in six sections. Section 1 provides an introduction while Sect. 2 provides a literature review of multimedia learning, e-learning, design elements in teaching videos, decision trees, and re-fined Kano model. The study’s methodology including implemental procedure and illustrative example are presented in Sect. 3. The results of the study are analyzed and discussed in Sects. 4 and 5, respectively. Finally, conclusion and possible future research directions are discussed in Sect. 6.

2 Literature Review

2.1 Cognitive theory of multimedia learning

In the cognitive theory of multimedia learning, Mayer [44] mentioned that compared with a single text, visually presented multimedia teaching materials (such as images, videos, animations, etc.) are usually more effective in learning and can be effectively transmitted complex information. The presentation of multimedia teaching materials is a key factor affecting the effectiveness of learning. Therefore, how to effectively combine teaching content with multimedia presentation is the goal of every instructional designer. The three basic principles in cognitive theory of multimedia learning are as follows:

  1. (a)

    Two-way pipeline: humans have independent sensory receiving channels, such as auditory processing of heard sounds and visual processing of images.

  2. (b)

    Limited capacity: human beings can only process limited information at any time, and cannot multi-task.

  3. (c)

    Active processing: when humans are learning, the coherent process of organizing and memorizing relevant data in the brain is extremely meaningful learning.

When designing textbooks, it is necessary to take the learner as the center and take the needs and feelings of the learners as the main considerations [52]. It is not an ignorant pursuit of new technologies, which causes the learners to be unable to adapt, produces cognitive load, and affects the learning effect. Cognitive load theory [55], mainly to help learners reduce cognitive load in the learning process. Cognitive load can be divided into:

  1. (a)

    External cognitive load: the cognitive load generated when using textbooks due to presentation methods that have nothing to do with learning goals.

  2. (b)

    Intrinsic cognitive load: the cognitive load that a learner generates when he understands the learning goal. The amount of this load depends on the difficulty of the content of the textbook.

  3. (c)

    Associated cognitive load: refers to the load generated by the additional notes or reference files made by learners in order to understand the content of the textbook.

In [44], he also mentioned that there are five ways to reduce external cognitive load, namely the principle of consistency, the principle of prompting, the principle of redundancy, the principle of spatial continuity and the principle of temporal continuity. Therefore, in the case of limited audiovisual capabilities and reduced cognitive load, it is necessary to combine design elements with learning objectives to help learners understand the knowledge content more easily.

2.2 E-learning

E-learning generally refers to the way of learning through electronic devices [5, 16, 42, 52, 62]. The three key points of e-learning materials are content, method, and purpose, which are described as follows:

  1. (a)

    Content: Content presented in e-teaching materials, including text, images, illustrations, diagrams, photos, animations, or movies.

  2. (b)

    Method: The media used in e-learning, such as desktop computers, notebook computers, tablets, smart phones, virtual reality, etc.

  3. (c)

    Purpose: Arouse learners’ motivation for autonomous learning.

E-learning has the characteristics of being able to watch, search, and learn whenever and wherever possible. Nowadays, the types of e-learning materials are very diverse. Learners can choose their own learning tools to improve learning efficiency and effectiveness.

Möller et al. [48] used teaching videos combined with role-playing for employee training. Their results showed that when a teaching video or role-playing has been used separately, both can improve learning effectiveness, but the difference between these two is not significant. If the teaching film can be combined with other methods, such as role-playing, it can dramatically increase the learning effect. Abachi et al. [1] also believed that the combination of film and other learning methods is more effective in learning. Recently, new technologies such as AR, VR and games [52, 59, 68] have employed to enhance the learning performance of e-learning. It can be seen that videos in e-learning materials are often used in combination with other learning methods. Therefore, this study will design eight elements into our teaching videos.

2.3 Design elements in teaching videos

Design elements could be considered as factors that visualize the content to be delivered in the film, including text, images, animation, sound, etc. [36, 66, 67]. Through meta-analysis, Höffler et al. [27] indicated that dynamic visual presentations can produce better learning effect than static visual presentations. Brame [8] mentioned that three elements need to be considered when designing instructional videos: (1) How to effectively use the film as a learning tool when watching instructional videos and reduce cognitive load; (2) How to maximize learners’ participation of the film; (3) How to motivate learners to learn independently when watching the film. Based on these three elements, Brame [8] provided the following suggestions for designing instructional videos:

  1. (a)

    The screen should be concise, with learning as the goal.

  2. (b)

    Appropriately match visualization and music to achieve complementary effects, rather than redundancy.

  3. (c)

    Use prompt messages to highlight important information.

  4. (d)

    The role’s lively and enthusiastic dialogue style can increase learners’ sense of participation.

  5. (e)

    Enhance autonomous learning motivation through role interaction, guiding questions or incorporating relevant information.

Augmented reality [4, 68], virtual reality [22], and other multi-media technologies have widely applied to enhance learners’ motivation and learning performance. But, for designing e-learning materials, related studies are diverse. They focus on specific element individually. For examples, Imhof et al. [29] utilized arrow tips. Ilgaz et al. [28] employed flicker and brightness contrast. Boucheix et al. [7] employed the dynamic spreading-color arrows. According to available literature, relatively few works aims to fully consider all related elements in design teaching videos. From current researches, “visual cueing” and “emotional design” [8] are the key for making instructional videos. Therefore, this study attempt to consider all elements of “visual cueing” and “emotional design” in the same time. Two dimensions, “visual cueing” and “emotional design”, of design elements have been introduced as follows.

2.3.1 Visual Cueing

“Cue” is one of the elements of visual communication in multimedia learning. It uses dynamic effects to attract the learner’s attention, thereby helping learners understand the knowledge conveyed by the textbook and deepen their memory. Consequently, in the process of “cueing”, how to attract the learner’s attention is the main key, such as increasing the brightness, changing the color, enlarging the font or changing it to bold, etc. to increase the visual effect of a specific element and highlight the importance of the element.

The so-called “visual cues” of videos are intended to make the information more prominent through various visual effects or methods, and make it easier for learners to obtain the information of the learning goals without adding new content [34, 43, 64]. Instructional videos often use visual cueing as a tool to help learners understand complex content (e.g. explanatory content, emphasized/reinforced instructions). Instructional designers can give corresponding visual cues for different learning goals to achieve different effects. In addition to the function of guiding learners to focus on learning goals, visual cueing can also enhance attractiveness and interest. In visual cueing methods, lots of studies often use “arrows” or “lines” combined with visual effects to present visual cues. For examples, Scholl [65] used flashing visual effects to attract learners’ attention and maintain their attention. Imhof et al. [29] utilized arrow tips between multiple pictures to enable learners to produce mental animations. Ilgaz et al. [28] employed flicker and brightness contrast to prompt in static images to attract learners’ attention. Boucheix et al. [7] employed the dynamic spreading-color arrows to explain the operation mode in the animation, which improves the understanding of learners. Table 1 summarized the types and applications of visual cueing used in related works.

Table 1 Related works in visual cueing
Full size table

The goal of “cueing” is to minimize cognitive load that has nothing to do with learning. Renkl [61] presented three points of view on visual cues will affect learners’ attention. They are prior knowledge, attention, and abstraction instead of simulation, detailed as follows:

  1. (a)

    Prior knowledge: The research of Reid et al. [60] shows that, compared with ordinary children who have never been in contact with this field, children who already have prior knowledge in this field can learn from pictures and texts more easily.

  2. (b)

    Attention: How to effectively enhance attention through visual cues, so that it is no longer just a pleasing function, but can achieve the effect of assisting understanding, which is the task of instructional designers.

  3. (c)

    Abstract instead of immersion: Realistic visualization with shapes, colors, textures, and details may cause distraction. Learners need to capture the correct learning target information in a short time. Excessive details may be not related to the learning target, and will lead to external cognitive load.

Although “cues” have the advantages of attracting attention and assisting viewers to quickly find the key points, they must also be used to properly control the frequency and method of use. Research on eye tracker tracking shows that students who are new to visual learning will pay special attention to the screen prominent element, but this element is not necessarily related to the learning goal [9, 25, 41]. This will cause learners to be distracted and ignore important information. Therefore, too many “cues” on the contrary will lead to the opposite effect.

Moreover, the concept of “seductive details” is intended to have interesting design elements that are not related to the learning goals [45]. In order to make boring courses attractive, previous researches will add “seductive details”, but Harp [24] stated that because of the limited memory capacity of humans, it is not recommended to use too many “seductive details” in multimedia. So, how to find relevant and learning goals elements that interest learners is a big challenge.

In summary, based on the above-mentioned literatures, this study uses visual cueing elements, including “arrow cueing”, “dynamic arrow cueing”, “spreading-color cues”, and “cueing method: contrary to cueing” to design questionnaire. Among them, “spreading-color cues” refer to [7], and we changed the application of color gradients from arrows to materials.

2.3.2 Emotional Design

“Emotional design” is a way to induce emotions. In the process of learning, “emotions” have a vital function [26, 33, 49]. Each design element will stimulate various emotions of learners. Emotions are mainly divided into positive emotions and negative. Positive emotions represent enjoyment, pride and satisfaction, and negative emotions represent anger, depression and anxiety. When learning tools can satisfy the learner’s heart, it will trigger positive emotions, which in turn will inspire learners and improve learning effectiveness [14, 47, 57]. In addition, positive emotions can increase attention, stimulate curiosity and creativity [20, 69].

Regarding negative emotions, Moridis et al. [49] and Fredrickson et al. [20] both show that negative emotions can also improve learning effectiveness. The negative emotion of anxiety represents the learner’s incomprehension of the teaching film, but it will prompt the learner to focus more on the teaching film and try to understand the content [51]. Table 2 lists the references for emotional design in this study. We will use these design elements to develop teaching film and design questionnaires based on this.

Table 2 Related works in emotional design
Full size table

The font and color of the text in the teaching film are also one of the factors that affect mood. Many studies have shown that when teaching videos use bright colors such as red, green and blue, or brighter colors such as yellow and orange, and the font is Kristen ITC, the size is between 15-24pt, which can arouse learners’ enthusiasm mood. In contrast, when using dim, low-brightness colors such as white, brown, and purple, and the font size is between 10-13pt, it will trigger negative emotions [17, 35, 39, 50]. Lohr [40] suggested that teaching designers use warm colors to make design elements more eye-catching, and clear and easy-to-read fonts and pictures can also help learners better understand the learning content.

Dalacosta et al. [13] applied anthropomorphic elements, such as baby face, big eyes, small nose, and high forehead to attract learners’ attention and trigger positive emotions. In the work of Kaya et al. [32], color has the energy to convey information and express emotions. It includes (1) color hue, saturation and brightness (2) color block size (3) foreground and background color contrast, proportion, etc. Raihan [58] and Uzun et al. [71] also believe that the contrast between the object and the background color can also improve attention.

In teaching videos, visual communication is usually accompanied by some sound effects, so that the teaching videos will not be boring [71], but the sound effects presented in the teaching videos must be related to the learning content. If the design element is purely entertaining, it will lead to “seductive details.” For example, when the scene of the crane moving goods appears in the teaching video, it must be accompanied by the sound of the crane starting, so that the learner can pay attention to the next behavior of the crane.

In the application of emotional design, we can use appropriate “visualization” to stimulate learners’ emotional responses to the teaching videos, but we must pay attention to maintaining the principle of “ensure that information can be accurately conveyed to learners”. The design must be visualized to balance with the cognitive load, overly complex animations will cause learners to need more sense of space and rhythm, which will increase the external cognitive load and affect the learning effect. Based on the above, this study will takes “font style”, “personification”, “color application”, and “audiovisual complementarity” as our design elements that affect “emotion”.

2.4 Decision Trees

Decision trees are algorithms of establishing a classification model through a tree structure [12]. Through the learning of training samples, classification rules are established, and then new samples are classified according to the classification rules. In this study, we use the C5.0 algorithm and the boosting method to improve the accuracy of the model. C 5.0 takes up less system resources and memory, so the calculation speed is faster.

Decision trees are not only easy to be used, but also can quickly find the knowledge rule for classifying complex data. At present, decision trees have been successfully applied in many fields, including education areas. For example, Kabra et al. [30] and Hamoud et al. [23] used decision tree methods to predict and analyze student behaviors. The experimental results show that students’ health, social activities, interpersonal relationships and academic performance will affect learning performance. Chen et al. [11] also used the decision tree method to conduct research on students’ employment wisdom courses, provide solutions for training professionals and employment courses, and solve the contradiction between training plans and enterprise needs. Al-Hoqani and Regula [2] built a semi-automated assessment model by using decision tree. However, most of these studies only focus on the behavior of learners, and do not analyze the teaching materials. Therefore, this study will apply the decision tree method to the in-depth analysis part of the video learning materials, and uses the decision tree to find the corresponding knowledge rules between the categorization of design elements and the user data of the teaching video.

2.5 Refined Kano Model

Kano et al. [31] were inspired by the two-factor theory of psychologist Herzberg, and proposed the well-known Kano Model, which can better understand and evaluate quality attributes. Recently, Kano model has been widely applied in lots of areas. For examples, Yu and Ye. [76] used Kano model to discover the users’ needs of automobile functions. Chen and Hsu [10] utilized Kano model to analyze users’ needs of e-learning. Lai and Lu [37] employed Kano model to discover the needs of dynamic display of clothing VR. Kano model already has successful application in lots of areas, except for instructional videos. Consequently, we also employed Kano model to analyze the VR exergaming users’ needs in this work.

However, the Kano Model does not take into account the importance of attributes. Therefore, Yang [74] proposed the Refined Kano Model, which expanded the 4 quality elements of the original Kano Model into 8 quality elements, and provided companies with more precise decision-making.

The quality elements of the Refined Kano Model are defined as follows.

  1. (1)

    High Attractive Quality: This attractive quality element is of high importance and has a strong appeal to customers. Therefore, the company needs to invest more in such quality factors as a weapon to attract customers.

  2. (2)

    Less Attractive Quality: The importance of this attractive quality attribute is low. Although such quality elements can attract customers, they have little effect. Whether the company needs to provide it continuously depends on the cost.

  3. (3)

    High Value-Added Quality: The importance of this one-dimensional quality attribute is high, and such quality elements have a significant contribution to customer value. Therefore, manufacturers should try their best to improve and maintain these quality factors to enhance customer value and loyalty.

  4. (4)

    Low Value-Added Quality: The importance of this unary quality element is low, so it does not contribute much to customer value. The company can determine the amount of such quality elements after measuring the cost. However, this quality element cannot cause shortage of supply and make customers dissatisfied.

  5. (5)

    Critical Quality: In addition to the essential quality, such quality factors are also highly important, so they are of considerable importance to customers. The company must try its best to maintain the degree of availability of this quality element

  6. (6)

    Necessary Quality: Although such quality elements are necessary qualities, their importance is low, so they are not the focus of customers' attention. Therefore, manufacturers only need to provide an appropriate level, but they need to avoid providing too low and soliciting customer dissatisfaction.

  7. (7)

    Potential Quality: This quality factor is temporarily indifferent, but its importance is showing signs of increasing. This phenomenon is not easy to occur, but it is possible, so the company cannot ignore this quality factor.

  8. (8)

    Indifferent Quality: This quality attribute is not only indifferent quality but also low in importance, so customers will not pay too much attention to this quality attribute. Companies can choose to eliminate such quality attributes in order to reduce costs in formulating strategies.

The Refined Kano Model has provided precise decision-making for many areas. For examples, Ferreira et al. [19] used multicriteria satisfaction analysis combined with Refined Kano Model to find ways to improve patient satisfaction. Through the categorization of key elements, they provide suitable strategies to increase patient satisfaction in Portuguese hospitals. Yeh et al. [75] used the Refined Kano Model, Quality Function Deployment (QFD) and Grey Relation Analysis (GRA) to improve the nursing capacity and competitiveness of nursing homes, and more importantly, the services to satisfy local residents. And the Refined Kano Model was used to find the key elements. Therefore, this study will also use the Refined Kano Model to find the key design elements of teaching films.

3 Methodology

3.1 Implemental Procedure

The implemental procedure of this study is mainly divided into two parts. In the first part, we use the Refined Kano Model to analyze, categorize the design elements that will appear in the instructional film, and find the expectations and needs of the learners. In the second part, this study will use decision tree to analyze learners with different backgrounds (frequency of viewing, whether they have experience in use, etc.) to find out the important factors that affect learners’ categorization of design elements.

The implemental procedure of used methods can be divided into the following six steps.

  • Step1: Define the design elements of the Refined Kano Model

Through reading literature (including e-learning, design elements, emotional design, etc.), we can define design elements of teaching film. In fact, 8 elements from two dimensions, visual cueing and emotional design, have been employed to design teaching film. To sum up, eight elements including “visual cues” and “emotional design” are designed in the teaching film (animation).

  • Step2: Make a teaching film and evaluate the defined design elements

This study uses the operation mode of the piano mechanism [7] as the content of the teaching film. In Fig. 1, we can find that the operation of piano mechanism can be divided into three stages: (1) hitting the key; (2) hitting the string with a hammer (3) bouncing back hammer, and the damper close to the string to stop the string. Finally, the mechanism will return to the original position and wait for the next action. The piano sound must use the operation of piano mechanism. However, the operation sometimes will have the problem of time overlap and rapid operation, which cannot effectively enable learners to understand the whole operation mode.

Fig. 1
figure 1

An example of teaching film- Steps of the operation in piano mechanism

Full size image

In instructional videos, we have to incorporate design elements into them. The design elements of teaching videos and presentations of teaching films are evaluated by experts in the field through expert interviews. According to its suggestions, we can adjust the presentation of the design elements in the film.

  • Step 3: Design Kano style questionnaire

In this step, we design Kano style questions based on defined design elements which have been modified in step 2. For each design element, we’ll design a pair of question items, the functional and dysfunctional questions. These Kano style questions are designed for Kano analysis. In addition, an importance question of the defined design element is also measured in this questionnaire for doing Re-fined Kano Model and constructing decision trees. To sum up, in this questionnaire, learners indicate if they feel satisfied or dissatisfied with a given situation. First, a situation supposes the design element is present or sufficient, and a second situation assumes the design element is absent or insufficient. Finally, they express the feeling about the level of importance for a certain design element.

  • Step 4: Collect data

Since respondents have to view teaching film first, they could answer our questionnaire. Therefore, the questionnaire will be issued and responded in the Surveycake platform. Before issuing, the designed questionnaire will be given to a small group of internet users for pilot testing. 30 participants are requested to respond the designed questionnaire online. Based on their responses, we can slightly modify the questionnaire. In 8 defined elements, we adjust the descriptions of questions, including changing terms and marking keywords in red.

Then, we attempt to collect some representative samples from the internet. In this study, those who have online learning experiences will be our survey target. Random sampling will be used. To make sure that we can get right samples, first, we organize an email list which contains almost 1,500 contact persons. From October 2019 to February 2020, we randomly sent requests to about 1,000 persons to respond our online questionnaire which can make sure they have experiences of seeing teaching films. Those learners might have experiences of e-learning will be requested to respond our questionnaire.

  • Step 5: Implement Re-fined Kano Model analysis

The main purpose of this step is to analyze the collected data to determine the categorization of elements. In this step, we analyze the collected data by Re-fined Kano model. By combining the two answers in the Kano evaluation table, we can categorize learner’s requirements into a category of Kano model. Based on the learner’s answers to Q1 (functional) and Q2 (dysfunctional), we can then find the categorization (Attractive, One-dimensional, Must-be, Indifferent, Reverse & Questionable) of that the design element according to Table 3.

Table 3 Kano evaluation table [31]
Full size table

Then, we have to masure the importance level. If one leaner’s importance score is higher than the average value, the element will be categorized into attributes of high importantce (High Attractive Quality, High Value-Added Quality, Critical Quality & Potential Quality). If one leaner’s importance score is lower than the average value, the element will be viewed as attributes of low importantce (Less Attractive Quality, Low Value-Added Quality, Necessary Quality, Indifferent Quality). Table 4 provides the comparison of quality element categorization between Kano and Re-fined Kano model.

Table 4 Quality element categorization in Kano Model and Refined Kano Model
Full size table

  • Step 6: Implement decision tree analysis

In this step, we attempt to extract advanced knowledge from the collected data. We use the background data of learners to predict their quality element categorization. The input variables include gender, age, design expertise, e-learning platform, usage length, usage frequency, and viewing time. And the output should be specific design element with its categorization. Then, we can construct decision trees. The detailed steps to build a decision tree are as follows.

  • Step 6.1: Determine the input and output variables of the decision tree

  • Step 6.2: Use C5.0 algorithm to construct a decision tree

  • Step 6.2.1: Establish the initial rule tree

  • Step 6.2.2: Prune this rule tree and increase readability

  • Step 6.2.3: Select the best performing rule tree

  • Step 6.3: Pick a tree whose performance is the best among all constructed trees

In this step uses fivefold cross validation experiment and construct a decision tree for each fold of data. In other words, the collected data set will be divided into 5 equal sized sets and each set is then in turn used as the test set. Beside test set, we use other 4 sets as our training set to build decision trees. Therefore, we will have 5 trees. Finally, the tree which has the best performance will be left for further analysis.

  • Step 7: Draw a conclusion

Finally, from the results of steps 5 and 6, we find the important design elements of teaching films, understand the learners’ thinks, and then we can draw conclusions based on them.

3.2 An Illustrative Example

Let’s take the element “arrow cueing” as an example. For this element, according to the definition of “arrow cueing” mentioned in Table 9, we can create two Kano style question items and one importance question item, as follows.

  • Q1: How would you feel if we use “arrow cueing” to mark the location of the key points, so that the learner can directly and quickly see the learning target that needs to be paid attention to (See Fig. 2a)? (A)Like (B) Must-be (C) Neutral (D) Live with (E) Dislike

  • Q2: How would you feel if we DON’T use “arrow cueing” to mark the location of the key points, so that the learner can directly and quickly see the learning target that needs to be paid attention to (See Fig. 2b)? (A)Like (B) Must-be (C) Neutral (D) Live with (E) Dislike

  • Q3: How would you feel the importance of “arrow cueing”? (A)Very important (B)Important (C) Neutral (D)Unimportant (E)Very unimportant

Fig. 2
figure 2

An example of question items of “arrow cueing” design element

Full size image

If one learner’s answers to Q1 (functional) and Q2 (dysfunctional) are “Like” and “Must-be”, respectively. According to Table 3, we can then find that the design element “arrow cueing” will be considered as an “attractive” quality factor for Kano analysis. Next, based on results of Kano analysis (attractive quality), we can get results of Re-fined Kano model based on Q3 (importance). From Table 4, if the answer of Q3 is larger than the average value of all samples for this element, the categorization of Re-fined Kano model will be “High Attractive Quality”. If the answer of Q3 is smaller than the average value of all samples, the categorization of Re-fined Kano model will be “Less Attractive Quality”.

Next, we use basic information of responders, including gender, age, design expertise, digital learning platform, the frequency of using instructional videos, and so on to be our input variables. Categorization results of Kano analysis of one specific element will be our output variable (class). In this case, we use results of Kano analysis of “arrow cueing” as our output variable. After implement decision tree algorithm, we can have three knowledge rules listed as below. From these rules, we can know male responder whose age is below 30 years old and have no design expertise; the length of their viewing video is less than 4 min; and think the importance of arrow cueing element is a little bit important. Then, they will consider “arrow cueing” element is “attractive”.

  • Rule #1

    IF Gender = Male AND Age < 30 AND Design expertise = NO

AND Average video viewing time <  = 4 Min AND Importance = 3 ~ 4.

THEN Arrow cueing = Attractive.

  • Rule #2

  • IF Age >  = 30 AND Average video viewing time = 4 ~ 20 Min AND Importance > 3.

  • THEN Arrow cueing = Attractive.

  • Rule #3

  • IF Design expertise = NO AND Frequency >  = 3 times per week AND Importance > 4.

  • THEN Arrow cueing = Attractive.

4 Implementation and Results

4.1 Quality factors of keyword advertisements

In this study, eight elements including “visual cues” and “emotional design” are designed in the teaching videos (animation). The element names and their academic supports are shown in Table 5. Next, the 8 designed elements will be reviewed by 5 domain experts.

Table 5 Design elements and their academic supports
Full size table

Table 6 provides their brief background including education and service length in design or online teaching areas. After reviewing designed elements in teaching animation, they provide some opinions and modification suggestions which are summarized in Tables 7 and 8. These five experts have many years of experience in the field of design or teaching, so they can give objective suggestions on the design elements in the teaching film.

Table 6 Background of domain experts
Full size table
Table 7 Expert suggestion about revising design element
Full size table
Table 8 Summary of 5 experts’ suggestions
Full size table

According to the comprehensive opinions given by the experts in Table 8, the experts believe that the presentation of visual cues in the teaching film needs to have a sense of rhythm, and the color matching of the overall picture is also very important. The font and typeface need to be fine-tuned according to the aspect ratio of the teaching film to clearly present the knowledge content. In “personification” part, they think that there is still the possibility of blurring the focus. Finally, eight design elements and their definitions, supports have been summarized in Table 9.

Table 9 The designed elements (functional) and their definitions
Full size table

4.2 The collected data

This study mainly investigates learners who have used instructional videos, and issue online questionnaires to allow respondents to watch the design elements of the video while answering question items. A total of 316 online questionnaires were collected. After manual examination, excluding invalid questionnaires with incomplete answers and contradictory questions, there are 309 valid questionnaires.

Table 10 shows the basic statistics of 309 respondents. In terms of gender distribution, “male” accounted for 38%, and “female” accounted for 62%. Regarding age, the proportion of 254 persons with “18–30 years old” accounted for the most 82%, followed by the proportion of “30–50 years old” with 34 persons accounting for 11%. The most commonly used digital learning platform is “YouTube” (93%). In terms of the distribution of "whether the expertise is related to design", 76 people with “yes” account for 25%, and 233 people with “no” account for 75%. Most people’s expertise is not design-related. About the question “How long have been using instructional videos?”, the proportion of “less than half a year” was 33%, followed by the proportion of “three years or more” accounting for 23%. Regarding “How often did you use the teaching videos to learn in the past month?” the number of “less than three times a week” was 215, accounting for 70%, followed by “three to five times a week” with 56 samples, accounting for 18%. About “the average length of time you watched an instructional video for learning in the past month”, the number of people with “4–20 min” accounted for 54%, followed by those with “4 min (inclusive) or less” accounted for 26%.

Table 10 Basic information of the collected samples
Full size table

4.3 The results

4.3.1 Refined Kano analysis

First, eight design elements were analyzed by Kano Model, and results have been summarized in Table 11. From this table, it can be found that the “dynamic arrow cueing” belongs to the “attractive quality”, which means that in the instructional video, the instructional designer will not make the learner dissatisfied even if the instructional designer does not provide this design element. But, when the instructional designer provided this element in video, it will increase learners’ satisfaction. “Font style” belongs to the “must-be quality”, which means that this design element must have but does not affect the increase of satisfaction, and not having it will increase the learner’s dissatisfaction with this design element. Besides, “arrow cueing”, “spreading-color cues”, “cueing method”, “color application”, “personification” and “audiovisual complementarity” have been categorized into “indifferent qualities,” which means that the presence or absence of these design elements does not affect learners’ satisfaction and dissatisfaction. And, we can find that most of the quality factors are indistinguishable, so we use the Refine Kano Model for in-depth analysis.

Table 11 Results of Kano & Refined Kano analysis
Full size table

Results of Refine Kano Model analysis also could be found in Table 10. We can find that “dynamic arrow cueing” are classified as “highly attractive quality”. This means that this design element is of high importance and can have a great appeal to learners. Therefore, instructional designers need to invest more in this design element as a tool to attract learners to watch instructional videos and guide correct learning goals. “Font style” is classified as “critical quality”. It means that in addition to the essential existence of this design element, it is also highly valued by learners. Consequently, instructional designers must try their best to maintain this design element. Additionally, “arrow cueing”, “spreading-color cues”, “color application” and “audiovisual complementarity” have been considered as “indifferent qualities.” It means that learners will not pay too much attention to these design elements, so instructional designers can choose to eliminate them to reduce costs. The “Cueing method” and “personification” are classified as “potential qualities.” This means that these two design elements are only temporarily indifferent quality. Due to the phenomenon of rising importance, teaching designers cannot ignore this design element.

4.3.2 Decision trees

In this study, eight variables, “gender”, “age”, “design expertise”, “most commonly used video learning platform”, “use of teaching video learning time”, “average use of teaching video learning frequency”, “average learning time spent watching a teaching video” and “importance” are used as input variables, and the Kano model element categorization results are used as output variables. Through decision tree analysis, in-depth analysis of the key conditions that affect the design element categorization. To have a stable performance, we implement tenfold cross validation experiment. Therefore, for each output variable, we construct 10 trees. Table 12 summarize the performances of decision trees in eight elements.

Table 12 Summary of results of decision tree with different output variables
Full size table

From Table 12, we could find the average accuracies are not good. Therefore, for each element, we only pick the tree with best performance for later analysis. And, the best classification accuracy of “cueing method” element is below 50% and it has been removed. The selected trees with the best performance are shown in Table 13.

Table 13 Discovered knowledge rules by using decision trees
Full size table

5 Discussions

According to the results of the stratification analysis listed in Table 14, we can find that design expertise and non-design expertise have some differences. “Design expertise” learners pay more attention to the balance of the overall picture, so for the “arrow cueing” and “cueing method,” they hope that the text and the visual cues will be balanced, so they think these two design elements are very important. “Non-design expertise” learners pay attention to whether they clearly receive the correct learning goals, so the lack of “arrow cueing” and “cueing methods” will not affect their dissatisfaction improvement, but proper use can help them to understand knowledge and increase satisfaction. Therefore, this study suggests that instructional designers mainly provide “cueing methods” for “design expertise” learners as a balance between visual impact and knowledge content, and occasionally provide “dynamic arrow cues” as a method to guide learning goals and increase satisfaction. For “non-design expertise” learners, we can mainly provide “font style” so that learners can acquire knowledge content more clearly, and occasionally provide “dynamic arrow cues” and “cueing methods” as methods to guide and focus on learning goals. This will increase satisfaction.

Table 14 Results of stratification analysis
Full size table

Regarding online learning experience, learners of “less than half a year” need to understand “how to quickly find learning goals” and “how to learn to watch teaching videos,” so they need “cueing methods” and “font style” to help learners. “More than half a year” learners have experience in using instructional videos, so they can directly focus on the correct learning goals. This means that the “cueing method” can help learners find learning goals more, but does not have it and does not affect learners to acquire knowledge. “Font style” can help explain that the design element must be used continuously. Therefore, this research suggests that instructional designers mainly use the “cueing method” for learners who are “less than half a year old” to focus on the correct learning goals, and use the “font style” to clearly explain the learning goals and methods. Designers can occasionally provide “dynamic arrow cues.” In addition to guiding learning goals, it can also greatly increase satisfaction. For learners of “more than half a year”, we should occasionally provide “dynamic arrow cues” and “cueing methods” as methods to guide and focus on learning goals can also greatly improve satisfaction. “Font style” is appropriately provided as an aid to explain the content of knowledge.

In terms of the frequency of using instructional videos, the needs of learners of “less than three times a week” and “more than three times a week” are also different. Learners with “less than three times a week” have a limited usage frequency per week, and each time they use it, they must pay attention to the learning goals directly and quickly. So, they hope to provide more “cueing methods” and “font styles” elements. The learners of “more than three times a week”, because of the high frequency of use, in addition to the knowledge content, will gradually be attracted by the learning-related design elements, so they will pay attention to the “cueing method” and “personification”. The “audiovisual complementarity” has a certain appeal but is not very effective. Therefore, this research suggests that instructional designers should mainly provide “cueing methods” for learners who have less usage frequency to focus on learning goals and provide “font style” element to clearly explain the knowledge content. And it could occasionally provide “dynamic arrow cues”, in addition to guiding learning goals, it can also greatly improve satisfaction. For learners who have high usage frequency, they can occasionally be provided “dynamic arrow cues”, “cueing methods” and “personification” elements, which not only have the effect of attracting attention, but also enable learners to acquire knowledge while increasing satisfaction. “Font style” element can be provided appropriately. As for “audiovisual complementary”, the decision to provide is based on time cost.

Considering the length of the video, it turns out that learners of “less than 4 min” and “more than 4 min” differ only in the element of “font style”. For learners of “less than 4 min”, under the time compression, it is more necessary to clearly explain the knowledge content. For learners of “more than 4 min”, when watching the video, there is enough time for the video to show the design elements, and the text description can assist in the explanation. Therefore, this research suggests that instructional designers mainly provide “font style” for learners “less than 4 min” so that the knowledge content can be fully explained, and “dynamic arrow cues” and “cueing methods” as guidance and focus on learning goals. The “dynamic arrow cues” and “cueing method” for learners of “more than 4 min” as a method to guide and focus on learning goals can also greatly increase satisfaction. “Font style” provides appropriate information that can help explain the complete knowledge content.

6 Conclusions

Teaching videos are part of e-learning content, and they are also the learning methods closest to people’s lives. When people use instructional videos for learning, they all hope to acquire knowledge quickly and clearly. Therefore, instructional designers play an important role between learners and teaching materials, including multimedia, animation, and video. These designers must use each design element carefully to maximize the effectiveness of teaching films, to combine learning objectives with multimedia teaching materials, to achieve interaction, and to convey teaching content to learners more effectively and accurately.

At present, most of the researches on multimedia teaching materials only discussed learning effectiveness or satisfaction. When one design element has learning effectiveness or is highly satisfied, the designer will be recommended to use this design element. However, the results analyzed only through learning effectiveness and satisfaction. They are not accurate enough. In addition, when design elements are considered by designers, they often fail to consider the views of learners, and the ideas of these users usually are the key to the success of e-learning materials. Therefore, this study attempts to use the Refined Kano Model and decision tree model to conduct a two-dimensional quality analysis and classification based on the learner’s satisfaction and importance of the design elements in the instructional video, and give designers instructions and suggestions.

In this study, we defined 8 design elements through analyzing literatures which is regarding e-learning, design elements, emotional design, etc. Through the online questionnaire, a teaching video which includes designed elements is presented so that learners can understand the design elements and respond. In results of Refined Kano Model analysis, we found that “dynamic arrow cueing” is classified as “highly attractive quality”. “Font style” is classified as “key quality”. “Cueing method” and “personification” are classified as “potential quality”. “Arrow cueing”, “Spreading-color cues”, “color application” and “audiovisual complementarity” are classified as “indifferent quality.”

Therefore, through the analysis of the Refined Kano Model, it is found that “dynamic arrow cues” are very important to learners. It can also improve learners’ satisfaction while guiding learning goals, indicating that instructional designers should make good use of this design element and become a weapon for teaching videos. The analysis result of “font style” shows that the configuration and style of this design element are of great importance to learners, which means that instructional designers must try their best to maintain the use of this element.

This study uses respondents’ basic data variables to conduct a stratification analysis, and gives advices to instructional designers for different types of learners, as shown in Table 15. Among them, “mainly provided” means that the learner’s satisfaction is directly proportional to the degree of provision and is of high importance. “Occasionally provided” means that when the design elements are not yet provided, it will not make the learners dissatisfied with the instructional video, but when the design elements are provided, the learner’s satisfaction can be greatly improved, because the learner does not expect this design element to help him gain knowledge more easily. And “appropriately provided” means that the learner believes that the design element must exist in the instructional video, but it does not need to be excessive if it is appropriate.

Table 15 Suggestions from results of stratification analysis in Refined Kano Model
Full size table

Table 16 summarizes the important variables extracted in knowledge rules by decision trees. The important variables affect the categorization of design element. For example, for “arrow cueing”, gender, age, design expertise, frequency, length, and importance are important variables. The information listed in Tables 15 and 16 could provide instructional designers valuable knowledge for designing good quality teaching films.

Table 16 The important variables in decision trees
Full size table

Some possible directions for future researches could be found as follows. Firstly, this study focuses on individual design elements, we can try to combine different design elements in the future to find more applications. Otherwise, future works could try different innovative design elements to enhance the learning performance of teaching videos. Secondly, this study uses a questionnaire survey to analyze the collected examples through the Refined Kano Model and decision tree to find the needs of learners. In the future, eye trackers could be used to find which kind of design elements can attract learners’ attention. Thirdly, virtual reality devices and social media could be involved to design more innovative teaching videos. Fourthly, future researches can provide games for quizzes after watching instructional videos to understand whether the design elements correctly guide learners to focus on the correct learning goals. Last but not the least, because of the COVID-19 epidemic, there will be more and more demand for distance learning. Consequently, we can also conduct research on the design elements applied to distance learning, and find out the design elements that can enhance learning performance.