Emotional Design in Concept Maps – No Support but Also No Burden

Introduction

Modern and global society is exposed to various information that we must filter, evaluate, and interpret. Most of them can only be assessed correctly when relationships are understood. Understanding concepts and their relationships (conceptual knowledge) are essential in everyday life, especially in learning contexts. This poses a significant challenge for students, especially for the subject of biology (Schmid and Telaro, 1990), as it tries to elucidate how organisms in supersystems interact with or influence each other and their environment (Reiners et al., 2018). The complexity of the biological subject matter causes high element interactivity which means that different aspects highly interact and cannot be learned in isolation (Chen et al., 2015). To help students obtain the required conceptual knowledge and handle the high element interactivity (Williams, 1998), concept maps are recommended for biology classes (Kinchin, 2000).

Concept maps are graphical tools that make knowledge and relationships between different elements of a system visible (Novak, 1990) and can be used for various purposes (e.g., planning, teaching, learning, and diagnosing; Schroeder et al., 2018). For the study of concept maps, Blankenship and Dansereau (2000) found that learners could experience a so-called map shock characterized by “bewilderment of not knowing where to start or how to penetrate the topography of the map” (p. 294). Researchers assume that these cognitive challenges could produce an adverse affective reaction that demotivates and inhibits learning (Dees et al., 1994). A possible solution to this problem could be to enrich the text-based nodes of the concept maps with illustrations (Alpert and Grueneberg, 2001). According to the cognitive theory of multimedia learning (Mayer, 2001), the combined presentation of illustration and text can reduce cognitive load (e.g., Levie and Lentz, 1982).

A new research field assumes that distinct design manipulations (emotional design) influence learners’ affective-motivational state (e.g., positive emotions in general, situational interest, intrinsic motivation, and enjoyment) and thereby foster learning (Um et al., 2012). Emotional design is characterized by human-like features, round shapes, and appealing colors. Several studies have been carried out to investigate the effect of emotional design on learning, affective state, and mental effort (Um et al., 2012; Plass et al., 2014; Park et al., 2015; Plass and Kaplan, 2016; Uzun and Yıldırım, 2018). Some findings support the assumption that emotional design evokes emotions and enhances learning (Um et al., 2012; Mayer and Estrella, 2014; Brom et al., 2018; Tien et al., 2018; Wong and Adesope, 2020), while other findings showed that emotional design did not impact emotions and learning (Park et al., 2015; Münchow and Bannert, 2019; Stárková et al., 2019). Overall, the meta-analysis by Brom et al. (2018) showed that emotional design improves retention, comprehension, and transfer. Furthermore, a reduced perceived task difficulty, a weak effect for positive affect, and no significant impact on mental effort were found. A recent meta-analysis supported these findings (Wong and Adesope, 2020). In emotional design research, mainly the positive scale of the positive and negative affect scale (PANAS) was used to investigate the affective impact of emotional design. Several other authors presume that the effect of emotional design can vary and may be dependent on factors such as context, learning domain, learning time, nature of the presentation, intensity of manipulation, or grade level (e.g., Münchow and Bannert, 2019; Stárková et al., 2019; Wong and Adesope, 2020).

The emotional design applied to concept maps has so far been little investigated. Tien et al. (2018) examined the effects of multidimensional concept maps (digital concept maps where videos, pictures, or sounds are linked to the concepts) on learning and the affective state in college students. They found that colorful and animated multidimensional concept maps lead to “higher learning well-being” and better learning than achromatic multidimensional concept maps (Tien et al., 2018, p. 1). However, it should be noted that Tien et al. (2018) interpreted the emotional design differently than Um et al. (2012). Rather than using anthropomorphisms, they applied daily-life advertisement music, animations, and images to induce emotions. Thus, it is unclear whether emotional design features in concept maps (incorporation of colorful and anthropomorphized images into the concepts) increase the learning efficiency of concept maps or whether it increases cognitive load and impairs learning.

We examined whether concept maps in emotional design reduce inhibitory perceived cognitive load, influence the perceived affective state, and thereby improve learning of junior high school students in an authentic learning environment. The present study was designed for junior high school students as there is a discrepancy between the evidence-based potential of concept maps to improve students’ learning and the actual use in schools (Kinchin, 2001).

Theoretical Framework

Concept Maps

A concept map is a graphical tool that represents knowledge in adomain (Novak, 1990; see Figure 1 for examples). External information (e.g., learning text and teaching) or own knowledge (retrieval practice; Blunt and Karpicke, 2014) guide the construction of a concept map. A concept map consists of textual elements (concepts) connected by labeled links that define their relationships. Arrowheads indicate the reading direction at one or both ends of the links. Combining at least two concepts connected via a link is called proposition and forms a meaningful unit (Novak and Gowin, 1984). If the concepts “chlorophyll” and “leaf pigment” are connected via the labeled arrow “is,” the proposition “chlorophyll is (a) leaf pigment” can be formed. The number of propositions of a concept map is unlimited, but fifteen to twenty-five concepts are usually necessary to create a (good) concept map (Novak and Cañas, 2008). Novak and Cañas (2008) recommend structuring concept maps hierarchically. The concepts are sorted according to their relevance and essential terms at the top and more specific terms below in the concept maps (Figure 1).

To explain why the construction and the study of concept maps are effective for learning, Schroeder et al. (2018) classified three reasons: (1) concept maps promote meaningful learning, (2) concept maps reduce extraneous cognitive load, and (3) concept maps do both.

While learning with concept maps, existing knowledge structures are modified and linked with new information (elaboration), a fundamental component of meaningful learning (Renkl, 2010). According to Großschedl and Tröbst (2018), concept maps help organize new information and promote metacognitive learning. Metacognitive learning is encouraged since concept maps help to reflect individual learning progress and identify knowledge gaps (Mintzes et al., 1997). For studying concept maps, it is assumed that learners adopt the (experts’/teachers’) knowledge structure and integrate the information of the concept map into their knowledge (Gehl, 2013). The learning effectiveness of concept maps of both processes (constructing and studying) has been empirically confirmed by meta-analyses (Horton et al., 1993; Nesbit and Adesope, 2006; Schroeder et al., 2018).

Emotions, Emotional Design, and Learning

There are two opposing hypotheses about the possible effects of emotions on learning: the emotions-as-facilitator-of-learning hypothesis and the emotions-as-suppressor-of-learning hypothesis (Um et al., 2012). The emotions-as-facilitator-of learning-hypothesis suggests that experiencing positive emotions can enhance learning performance. According to Um et al. (2012), “experiencing positive emotions during the learning process can enhance learning outcomes, either through direct impact and learning or through mediating variables, such as interest and motivation.” Furthermore, (design) elements that elicit positive activating affective states are assumed to provoke deeper cognitive processing and thereby increase the cognitive resources dedicated to a task (see Stárková et al., 2019 for an explanatory graphic). By increasing available cognitive resources, aspects like affect and motivation can mediate the learning process at a higher level.

On the contrary, the emotions-as-suppressor-of-learning hypothesis postulates an impairing effect on learning. In this case, emotions are presumed to distract students from a task by, for example, processing information about one’s emotional state or other task-irrelevant thinking. This additional process competes for the limited capacity of the working memory (see Plass and Kalyuga, 2019 for a recent overview of the effect of emotions on cognitive processes).

Since studies on emotional design focus on the learners’ general emotional state rather than specific types of emotions, in this study, the generic term “affect” is used to refer to emotion and mood (for a similar approach, see Plass et al., 2014; Park et al., 2015; Uzun and Yıldırım, 2018). According to the circumplex model of affect by Russell (2003), emotions can be categorized in a two-dimensional system with valence as a continuum from positive to negative (affect) as one dimension and activation as a continuum from activated to deactivated as the other. Activating emotions are enjoyment or curiosity (positive) and anger (negative). Deactivating emotions include relaxation (positive) and sadness (negative; Pekrun et al., 2006) and are mainly related to an impairment of learning (e.g., Aspinwall, 1998). Empirically, positive affect is typically found to increase interest and motivation to learn, while negative affect is mainly shown to impair learning [see meta-analysis by Barroso et al. (2021)]. On the contrary, beneficial effects on learning for some negative emotions and inhibitory effects for positive affect has been found (Münchow and Bannert, 2019). In a recent study by Mensink (2021), positive affect negatively predicted learning performance, while negative affect positively affected learning. In line, learners in a negative affective state before learning outperformed those in a positive affective state (Knörzer et al., 2016). Positive activating emotions are suspected of distracting from the learning material and acting detrimentally. Pekrun (2021) concluded that the type of effect (beneficial or inhibitory) of different emotions depends on, for example, the situation, the kind of task, and personal traits. Also, Knörzer et al. (2016) suspected that moderating trait variables are responsible for the inconsistent results. Beyond, it was found that the experience of multiple emotions (here, joy, anger, confusion, and frustration) could distract and negatively impact learning (Dever et al., 2021). The authors suspect that the processing of emotions causes additional cognitive load, reducing cognitive resources available for the learning task.

For multimedia designs, the effect of a design on the viewer [(visual) appeal], has been found to influence the affective state (Capota et al., 2007). Emotional design manipulations are developed based on this assumption, however, their effect on appeal has been little investigated so far (Mayer and Estrella, 2014). Contrary to expectations, learning material in emotional design was not perceived as more appealing compared to non-emotional design material (Mayer and Estrella, 2014). Besides examining the affective parameter enjoyment, Mayer and Estrella (2014) also assessed the desire for additional lessons of a similar nature which served as a proxy for positive affect. No difference between emotional design and non-emotional design material was found here either.

Cognitive Load and Multimedia Learning

In emotional design research, different theories are used to explain how emotional design influences learning and cognitive processes. Central to learning is the cognitive load theory (Sweller, 2005). It is assumed that the amount of information that can be processed simultaneously is restricted while different aspects can increase a person’s cognitive load (limited capacity assumption; Mayer, 2001). Sweller (2005) distinguishes between three types of cognitive load, i.e., intrinsic, extraneous, and germane load. The intrinsic load is caused by the task or learning material and depends on the complexity of these and the learners’ expertise. Consequently, it cannot be manipulated directly by instructional design. Germane load refers to the resources that learners invest in their learning process. In contrast, the extraneous load is not conductive for learning and arises from a poor learning material design (e.g., unsuitable design). A seductive detail, for example, is assumed to impose extraneous cognitive load and can impair learning by competing with the learning object for cognitive resources (Harp and Mayer, 1997; Lenzner et al., 2013).

According to the dual-channel assumption by Paivio (1986), information is processed in verbal and visual channels. Based on this assumption, the cognitive theory of multimedia learning (Mayer, 2001) was developed, which describes multimedia as any material that consists of a combination of text and illustration elements. It is assumed to improve learning by reducing extraneous cognitive load. The text can be written or spoken, and illustrations are understood to include static pictures (e.g., illustrations, graphics, and photos) and moving pictures (e.g., videos and animations). The cognitive theory of multimedia learning is mainly focused on cognitive processes, but it was shown that emotional and motivational aspects and cognitive processes are inextricably linked (Plass and Kalyuga, 2019). The cognitive-affective theory of learning with media was established to extend the previous theory with non-cognitive elements (emotional and motivational aspects) which can influence the learning process (Moreno and Mayer, 2005, 2007; Moreno, 2006). It presumes that affect and motivation can mediate the learning process by manipulating the cognitive resources available for a specific task.

Two competing views exist regarding emotional design and its influence on extraneous cognitive load. As stated in section “Emotions, Emotional Design, and Learning,” emotions induced by emotional design could cause extra processing. This process is not considered relevant for the learning goal, it can be classified as extraneous cognitive load (Plass and Kalyuga, 2019). Another potential source of extraneous cognitive load is that to achieve an emotional design, certain features must be added to the learning material. According to the coherence principle by Mayer and Fiorella (2014), design elements that are described as interesting and entertaining but irrelevant in terms of content could cause extraneous cognitive load. However, this applies primarily to seductive details (seductive detail effect, Park et al., 2011), which does not include the emotional design according to Um et al. (2012). The emotional design does not consist of irrelevant, decorative elements. Instead, elements of the learning material are equipped with emotional features (Um et al., 2012). Thereby unnecessary extraneous cognitive load should be avoided. In addition, the emotional design is not expected to increase the extraneous cognitive load, as it is assumed that it increases the cognitive resources that are made available for a task (as stated in section “Emotions, Emotional Design, and Learning”). Another aspect suggests that emotional design does not increase the extraneous load but even reduces it. This is because emotional design is supposed to guide attention (Park et al., 2015; Le et al., 2021; Peng et al., 2021). Search processes in a learning task are one source of extraneous cognitive load (Plass and Kalyuga, 2019), which can be reduced by guiding learners’ attention. Supporting this, students perceived the learning material adopting emotional design features to be less complicated (lower perceived task difficulty) than differently designed material [see meta-analysis by Brom et al. (2018)]. The perceived task difficulty represents a persons’ estimation of effort needed to manage a certain task (Schneider et al., 2021). For emotional design features, a low perceived task difficulty is expected as they are supposed to “being perceived by the students as easy to learn because of their entertaining appearance” (Uzun and Yıldırım, 2018, p. 125).

In contrast, incorporating emotional design elements should not affect intrinsic cognitive load. Since the different design features are not presumed to affect aspects directly related to the complexity of the learning content. However, Le et al. (2021) reported that the emotional design decreased intrinsic load. Emotions, in general, could either increase or reduce the intrinsic cognitive load, which is context-dependent, according to Plass and Kalyuga (2019).

Regarding the germane cognitive load, it can be presumed that the emotional design, which potentially increases cognitive resources and lowers extraneous cognitive load, could increase the resources available for meaningful, effective learning (increase germane cognitive load).

Hypotheses

This study examined the influence of emotional design on learning performance, cognitive load, and affective state in a two-experiment approach. As part of experiment 1, the effect of emotional design on junior high school students’ learning performance, different dimensions of perceived cognitive load (e.g., in the form of task difficulty), and perceived affective state (e.g., appeal, enjoyment, and desire for additional tasks of a similar nature) were investigated. In experiment 2, the change of the perceived affective state was investigated in more detail. Concept maps with (1) bright and saturated colors as well as anthropomorphisms (emotional design, ED), (2) grayscale coloring and non-anthropomorphic illustrations (neutral design, ND), and (3) without pictorial elements (control design, CD) were used.

Based on the theoretical foundation and previous research, the following hypotheses arise:

We expect students who study with concept maps in emotional design to outperform students of the other conditions in terms of learning performance (H1). We further assume that students in the emotional design condition experience lower extraneous cognitive load (H2.1) and higher germane cognitive load (H2.2) compared to the other conditions. No difference is expected for the intrinsic cognitive load (H2.3). We presume students in the emotional design condition to experience a lower task difficulty (H2.4) and rate the appeal, enjoyment, and desire for additional tasks of a similar nature as higher (H3.1) than the other conditions. In terms of affect, we assume concept maps in emotional design to induce a rise in positive affect and a decrease in negative affect (H3.2).

General Method

Two experiments were carried out in students’ regular classrooms. For all experiments, the same learning material was used, and all students underwent concept map training to ensure comparable knowledge in the use of concept maps. Two investigators carried out the studies, one presenting the instruction and the other providing individual support if needed. Teachers were welcome to attend the lessons. All students were informed about the aims and course of the investigation, the option to stop participating at any time, guaranteed protection of data privacy, and the no-risk character of study participation. Written consents from parents and school principals were obtained before the study. During the experiments, we comply with the requirements of the school law, which, for example, determines that the learning material corresponds to the teaching specifications and does not promote a discriminatory understanding of the students (North Rhine-Westphalian Ministry of Education Science and Research, 2020) and followed the ethical principles and guidelines for the protection of human subjects of research (World Medical Association, 2001).

Learning Material

The learning material covered the topic ecosystem and consisted of three double pages (DIN A3) presented in a study book. The ecosystem is a central topic in biology lessons (Lohmar and Eckhardt, 2014) and is well suited for the use of concept maps due to many possible interrelationships. Each double-page of the study book contained one of three concept maps referring to the aspects “Living Organisms in a Lake,” “Zones of a Lake,” and “Limnetic Zones in a Lake.” These concept maps followed the general principles of Novak and Cañas (2006), were hierarchically structured, and consisted of 16–25 concepts and 28–36 propositions. CmapTools software was used to construct the concept maps (available at https://cmap.ihmc.us/). Three study books were developed, each of them applied specific design principles (i.e., emotional, neutral, or control design) to the concept maps (see Figure 1).

The concept maps in the emotional design condition were constructed using bright and saturated colors and illustrations with anthropomorphisms based on the cognitive-affective theory of learning with media and along the lines of previous studies (e.g., Mayer and Estrella, 2014). Additionally, semantically related concepts were visually grouped by color (see signaling principle; Mautone and Mayer, 2001; for a similar approach see Nesbit and Adesope, 2011). For example, all concepts related to the main concept “Producers” (e.g., “Photosynthesis,” “Plants/Algae,” and “Sunlight”) were colored green. Illustrations without emotionally relevant design features were incorporated for the neutral design condition, and monochromatic grayscale was used to group related concepts. Plain concept maps without any illustrations or design manipulations served as control design.

The conditions did not differ in content or structure (e.g., the number of objects shown, their orientation, location, or size). To avoid attention being divided (see split-attention effect; Mayer, 2001) for the neutral and emotional design concept maps, the illustrations were always presented in close spatial proximity to the textual element. Complete concept maps on “Living Organisms in a Lake” are available as supporting information (see Supplementary Learning Material).

Concept Map Training

As training is recommended for the correct handling of concept maps (e.g., Holley and Dansereau, 1984; Allen and Tanner, 2003), we implemented a concept map training (135 min; Lenski et al., (in preparation)¹) before learning with concept maps. First, declarative knowledge about concept maps was conveyed (e.g., “What are concept maps?,” “What can they be used for?,” “What elements are concept maps made of?”). Learners were then provided with procedural knowledge. The training material was not specific to any school subject and covered various topics (e.g., “National Economy” and “Different States of Water”).

Data Analysis

Normal distribution was identified visually and tested statistically using histograms and Kolmogorov–Smirnov tests. Levene’s tests or Box’s M tests were conducted to check for homogeneity of variances. Unless stated otherwise, all assumptions were met in the respective analyses. Parametric and non-parametric techniques were implemented, and if not otherwise specified an alpha significance level of 0.05 was set for all analyses and hypothesis testing.

Experiment 1

Method

Design and Procedure

An experimental intervention study was implemented with a pre- and post-test design. The study consisted of six lessons (á 45 min) delivered over 3 weeks (two lessons a week). In the first lesson, prior knowledge, reading fluency, and demographic data (age, gender, and grade in biology) were determined by questionnaires (45 min). In the following three lessons, the students received training in concept maps (see section “Concept Map Training”). In the learning phase (45 min), study books in the emotional design, neutral design, or control design were randomly distributed among the students (see section “Learning Material”). The students were told they would be tested on the material’s content afterward and were instructed to study each concept map carefully. After the learning phase, cognitive load, task difficulty, appeal, enjoyment, and desire for additional tasks of a similar nature were assessed by self-rate questionnaires included in the study books. Puzzles were handed out to those who completed their learning phase early to avoid disturbing others. In the following lesson, learning performance was measured using a post-test questionnaire (45 min).

Sample

An a priori power analysis was performed for sample size requirements. Based on the findings of former studies (d_recall = 0.32, d_transfer = 0.33, d_{perceived difficulty} = −0.21; Brom et al., 2018), we expected a medium effect (cf., Cohen, 2013) of our treatment on our primary dependent measure (learning performance). Power analysis was conducted using G*Power 3.1.9.2 with the assumed medium effect (f = 0.28) and an alpha level of 0.05 at a power of 0.95. The result showed that a total sample of 223 students is required. In this context, Cunningham and McCrum-Gardner (2007) recommend estimating the drop-out rate in advance and adjusting the sample size. As to our knowledge, there are no benchmarks available on drop-out numbers from multi-day intervention studies in schools, we assumed a drop-out rate of about 15% based on previous studies of similar manner.

A total of 257 8th-grade students were recruited from four different German high schools and N = 202 students completed the entire experiment (47% female; age: M = 13.18, SD = 0.49; maximum classroom size: 33 students). All classes received 50 euros as compensation. Students who did not attend the training, the learning phase, or the post-test were excluded in further analyses (for detailed information, see the flowchart of participants in Figure 2).

Instruments

Reading Fluency

As reading fluency was identified as a predictor for successful reading (National Institute of Child Health Human Development, 2000) and is known to influence learning performance (Bigozzi et al., 2017), we captured students’ reading fluency by the Salzburg Reading Screening SLS 2-9 (Wimmer and Mayringer, 2014), a classroom-administered standardized reading speed task [test-retest reliability for normative sample (grade 8) r = 0.87].

Prior Knowledge

Since prior knowledge can account for a large portion of a students’learning performance (Shapiro, 2004), we controlled for prior knowledge on “Ecosystems” (including “The Lake Ecosystem”). Prior knowledge was measured by a pre-test questionnaire consisting of 30 items that include general items about ecosystems and specific questions about the lake ecosystem. A general item deals, for example, with the question of why photosynthesis is important for humans (see Supplementary Tests, p. 2, item 3). An item focusing on the lake ecosystem covers, for example, the role of decomposers in the ecosystem lake (see Supplementary Tests, p. 6, item 14). Three of the items in the pretest were taken from the third International Mathematics and Science Study TIMSS (Harmon et al., 1997; Baumert et al., 1998), as the items fit thematically well and have been validated for the 8th grade (see Supplementary Tests, items taken from the TIMSS study are marked accordingly). Nine items were excluded due to low items’ discrimination <0.20 (Durrheim and Tredoux, 2004). The remaining 21 items are single choice (13 items), multiple choice (4 items), and matching tasks (4 items). The complete questionnaire is available as supporting information (see Supplementary Tests). One point for each correct answer and zero points for incorrect answers were awarded; hence, the students’ total scores obtained could range from a minimum score of zero to a maximum score of 21 points (Cronbach’s α = 0.72).

Perceived Cognitive Load

Regarding cognitive load, mostly the self-rating items on perceived effort (Paas, 1992) and perceived (task) difficulty were used in previous studies. According to Brom et al. (2018), these are proxies (subcomponents) of cognitive load and may not be suitable for this purpose. Therefore, contemporary cognitive load questionnaires are highly recommended for future emotional design research (Brom et al., 2018). Based on this, we decided to use the first version of the naïve rating questionnaire by Klepsch et al. (2017) to measure the three types of cognitive load separately (seven items). The items were not modified from the original except for the replacement of “the task” with “the concept map.” Items were rated on a 7-point Likert scale from “not at all true” to “very much true,” with only the ends of the scale being labeled. Two items assessed the intrinsic cognitive load (“For this concept map, many things needed to be kept in mind simultaneously”; “This concept map was very complex”; Cronbach’s α = 0.51). The germane cognitive load was assessed by two items as well (“For this concept map, I had to engage myself highly”; “For this concept map, I had to think intensively about what things meant”; Cronbach’s α = 0.67). The extraneous cognitive load was investigated by three items (“When looking at concept maps, it was exhausting to find the critical information”; “The design of the concept map was very inconvenient for learning”; “When looking at concept maps, it was difficult to recognize and link the crucial information”; Cronbach’s α = 0.77).

Perceived Task Difficulty

To determine the perceived task difficulty, a single item was applied, which was used in previous studies (Um et al., 2012; Mayer and Estrella, 2014; “Please rate how difficult the concept maps were for you.”). This item was rated on a 5-point Likert scale from “very easy” to “very difficult.”

Self-Assessed Handling of Concept Maps

To determine whether all students had successfully passed the concept map training, they were asked to assess their handling of concept maps during the learning phase. The questionnaire was developed based on a self-regulation questionnaire in the context of a computer-based concept map training program for 10th-grade students (den Elzen-Rump and Leutner, 2007). It consists of four items on a 3-point Likert scale (e.g., “Did you look carefully at the arrow directions in the concept map?” on a scale from “seldom” to “very often”; Cronbach’s α = 0.70). As the original questionnaire was designed for a construction task, the items were modified for an observation-based task (e.g., “arrows drawn” was replaced with “looked carefully at the arrow directions”). Moreover, the emoticon-based rating scale was replaced by a number-based Likert scale. According to Pollock et al. (2018), emoticons should be used with caution in rating scales due to possible interpretation problems.

Perceived Appeal, Enjoyment, and Desire for Additional Tasks of a Similar Nature

To investigate affective parameters, we followed the approach of Mayer and Estrella (2014). They used a five-item questionnaire on task difficulty, mental effort appeal, enjoyment, and desire for additional tasks of a similar nature on a 5-point Likert scale. As in the present study a questionnaire on cognitive load is applied, only the four items covering appeal, enjoyment, and desire for additional tasks, and perceived task difficulty (see section “Perceived Task Difficulty”) were adopted. The question on mental effort was excluded to avoid redundancies. All items were translated into German and the term “the lesson” was replaced by “the concept maps” (appeal: “Please rate how appealing the concept maps were for you” on a scale from “very unappealing” to “very appealing”; desire for additional tasks of a similar nature: “I would like to learn from more concept maps like these” on a scale from “strongly disagree” to “strongly agree”; enjoyment: “I enjoyed learning from concept maps” on a scale from “strongly disagree” to “strongly agree”).

Knowledge of the Lake Ecosystem (Learning Performance)

Learning performance was measured by a 21-item post-test focusing on the “Lake Ecosystem.” Items were taken from the pre-test with two items slightly changed. Three items were excluded due to poor item parameters (items’ discrimination <0.20; Durrheim and Tredoux, 2004). The final version of the questionnaire includes 15 single choice items and three matching items. The full questionnaire is available as supporting information (see Supplementary Tests). The maximum score on the pre-test was 18 points (Cronbach’s α = 0.78).

Results

Preliminary Analyses

We checked whether the three conditions (emotional design, neutral design, and control design) differed in age, reading fluency, prior knowledge about ecosystems, grades in biology, and gender distribution. Since we assume that the null hypothesis is true for preliminary analyses, an alpha level of α = 0.10 is set to increase test power and reduce the probability of committing a type II error (Döring and Bortz, 2016). One-way analysis of variance (ANOVA) did not result in any statistically significant differences between conditions in age, F(2,161) = 0.23, p = 0.798, reading fluency, F(2,151) = 1.38, p = 0.255 or prior knowledge about ecosystems, F(2,151) = 1.38, p = 0.255. A chi-squared test revealed no statistically significant differences in gender, χ²(4) = 6.23, p = 0.182. Kruskal–Wallis tests showed that students did not differ referring to self-assessed handling of concept maps, χ²(2) = 0.55, p = 0.761, and grades in biology,χ²(2) = 2.02, p = 0.364. For descriptive data, see Supplementary Table 1. No unusual data distribution or outliers were observed.

Hypothesis Testing

Knowledge of the Lake Ecosystem (Learning Performance; H1)

We assumed that students in the emotional design condition outperform students in the neutral and control design conditions in terms of learning performance. A one-factorial analysis of covariance (ANCOVA) showed that, after adjustment for prior knowledge and reading fluency as covariates, the design conditions had no influence on knowledge of the lake ecosystem tested by means of a post-test, F(2,141) = 0.23, p = 0.796 (lack of support for H1). Table 1 displays mean values and standard deviations.

Cognitive Load and Task Difficulty (H2.1–2.4)

We expected the emotional design to affect the perceived cognitive load. The results of a bivariate correlation between all dependent variables showed moderate correlations between the cognitive load subscales (intrinsic, extraneous, and germane load) and task difficulty [bivariate correlation is available as supporting information (see Supplementary Table 2)]. The results together with the classification of task difficulty as a proxy for cognitive load (Brom et al., 2018), a one-way multivariate analysis of variance (MANOVA) was conducted. Design was set as independent variable and the cognitive load subscales as well as the perceived task difficulty as dependent variables. MANOVA assumptions were met, except for the multivariate assumption of normality. Since MANOVAs are robust with respect to those deviations from normal distribution (Olson, 1974), we use the original MANOVA (for a similar argumentation see Stárková et al., 2019). Although prior knowledge correlates significantly with cognitive load, it was not found to be a significant covariate [Wilks’ Lambda = 0.96, F(2,159) = 1.91, p = 0.13]. Thus, the variance analysis was performed without any covariate. A significant effect for design [Wilks’ Lambda = 0.90, F(2,201) = 2.60, p = 0.009, η²_p = 0.051] was accompanied by significant effects using univariate variance analyses (ANOVAs) on extraneous cognitive load, F(2,181) = 3.89, p = 0.022, η²_p = 0.038; intrinsic cognitive load, F(2,201) = 4.162, p = 0.017, η²_p = 0.040; germane cognitive load, F(2,201) = 3.16, p = 0.045, η²_p = 0.031, and task difficulty, F(2,201) = 4.32, p = 0.015, η²_p = 0.042.

Bonferroni post hoc analyses were conducted. In the neutral design, intrinsic cognitive load was lower compared to the emotional design, p = 0.050 (M_Diff = 0.56, 95%-CI [1.12, 0.00]), and compared to the control design condition, p = 0.031 (M_Diff = 0.60, 95%-CI [1.16, 0.04]; lack of support for H2.2). See Table 1 for mean values and standard deviations. The extraneous cognitive load was significantly lower in the neutral condition compared to the control design condition, p = 0.018 (M_Diff = 0.72, 95%-CI [1.34, 0.09]; lack of support for H2.1). The same pattern was observed for the germane load, p = 0.049 (M_Diff = 0.61, 95%-CI [0.00, 1.22]; lack of support for H2.3). The perceived task difficulty was highest for the control design condition and lowest for the emotional design condition, p = 0.015 (M_Diff = 0.49, 95%-CI [0.07, 0.91]; support for H2.4).

Appeal, Enjoyment, and Desire for Additional Tasks of a Similar Nature (H3.1)

Independent Samples Kruskal–Wallis tests showed that the students’ perceived appeal (of the concept maps), H(2,199) = 1.09, p = 0.58, enjoyment, H(2,201) = 1.99, p = 0.37, and desire for additional tasks of a similar nature, H(2,200) = 0.71, p = 0.70 did not differ significantly between conditions (lack of support for H3.1; see Table 1).

Experiment 2

Method

Design and Procedure

To investigate whether concept maps in emotional design influence positive affect, an experimental intervention study with a pre- and post-test design was conducted. First, demographic data (age, gender, and biology grade level) were determined, and students completed a questionnaire on positive and negative affect for the first time. Each student was then randomly assigned to one of the three study books implementing emotional, neutral, or control design (see section “General Method”). In a learning phase, students were asked to study the concept maps carefully.

Sample

An a priori statistical power analysis using G*Power 3.1.9.2 was performed for sample size requirements. For studies by Um et al. (2012) and Plass et al. (2014), large effect sizes of d_{PositiveAffect} = 0.79 and d_{PositiveAffect} = 0.88 were reported but in the meta-analysis by Brom et al. (2018) a marginal effect for positive affect (d_{PositiveAffect} = 0.11) was found. Since in the meta-analysis by Brom et al. (2018), studies were included in which only the color of a learning environment was varied, we consider the effect sizes gained by Um et al. (2012) and Plass et al. (2014) to be a better guide value for our study. A compromise was chosen, and a medium average effect size (f = 0.30) was anticipated. Accordingly, a total sample of 48 students was required for repeated measures ANOVA at a significance level of α = 0.05 at a power of 0.95. A total of 54 students (grade 9) were recruited from a German high school. Since n = 7 were excluded because they missed the concept map training, N = 47 students were assigned to the treatment conditions (47% female; age: M = 14.34, SD = 0.56).

Instrument

The Positive and Negative Affect Scale (PANAS; Watson et al., 1988) was used in the German version (Breyer and Bluemke, 2016) to measure affect. For positive affect, ten different emotions related to positive affect (interested, excited, strong, enthusiastic, proud, alert, inspired, determined, attentive, and active) were presented to the students (Cronbach’s α = 0.79). For negative affect, ten different adjectives describing negative affect (distressed, upset, guilty, scared, hostile, irritable, ashamed, nervous, jittery, and afraid) were presented (Cronbach’s α = 0.87). Each subscale was rated on a 5-point Likert scale from “not at all” to “very much.” Ratings are averaged by mean as recommended by Breyer and Bluemke (2016).

Results

Data were analyzed using IBM SPSS Statistics (version 26) and nparLD R software package (The R Foundation for Statistical Computing, Vienna, Austria).

Preliminary Analysis

Prior to hypothesis testing, the three conditions (emotional design, neutral design, and control design) were checked for differences in age, grades in biology, and gender distribution. As described earlier, for preliminary analyses, an alpha level of α = 0.10 is set as recommended by Döring and Bortz (2016). A chi-squared test revealed no statistically significant results for gender distribution, χ²(2) = 0.83, p = 0.659. Furthermore, it was shown by means of Kruskal–Wallis tests that students in all conditions did not differ in terms of grades in biology, H(2) = 2.31, p = 0.891, or age H(2) = 0.02, p = 0.990. No unusual data distribution or outliers were observed.

Hypothesis Testing

Affective State (H3.2)

We expected students to experience more positive affect and less negative affect when learning with the emotional design compared to the neutral or the control design. As the data were not normally distributed, a non-parametric counterpart of the repeated measures analysis of variance was conducted [rANOVA; nparLD R package; see Noguchi et al. (2012) for more information] with design as an independent variable and positive affect as a repeated measures variable. Descriptive statistics are shown in Table 2. We found no significant main effect for time of measurement (pre vs. post), F(1,47) = 0.01, p = 0.921. Also, no significant difference for the design could be determined, F(2,47) = 2.45, p = 0.087. Furthermore, time of measurement and design did not interact with each other, F(2,47) = 1.11, p = 0.325. To examine differences in the experienced negative affect, a rANOVA was calculated with design as an independent variable and negative affect as a repeated measures variable. No significant effect for the time of measurement, F(1,47) = 1.00, p = 0.315 and non for the interaction (time of measurement: design) was observed, F(1,47) = 0.55, p = 0.577. However, a significant main effect for the design could be found, F(1,47) = 4.82, p = 0.009. Post hoc analyses (Bonferroni) showed that learning with concept maps in the control design led to higher negative affect compared to concept maps in emotional design, F(1,47) = 4.36, p = 0.037 and neutral design, F(1,47) = 7.59, p = 0.006 (partly support for H3.2).

General Discussion

The purpose of this research was to examine the effect of emotional design in concept maps on junior high school students’ learning performance, perceived cognitive load, and perceived affective state. The concept maps covered the biological topic of the lake ecosystem.

Learning Performance

Based on the cognitive-affective theory of learning with media and previous findings (e.g., Um et al., 2012), we assumed concept maps with emotional design to influence learners’ affective-motivational state and thereby foster learning. Contrary to our assumption, emotional design did not foster learning performance. Our results contradict previous findings (Um et al., 2012; Mayer and Estrella, 2014; Brom et al., 2018; Tien et al., 2018; Wong and Adesope, 2020) but as current results in emotional design research are heterogeneous, our results complement other findings showing that emotional design did not influence recall, transfer, or comprehension (Park et al., 2015; Uzun and Yıldırım, 2018; Stárková et al., 2019). There are several possible reasons to explain conflicting results in the field. Varying intensity of emotional design could be one possible cause (Park et al., 2015) as features of emotional design could include variations in shape, color, and sound, as well as the use of anthropomorphic graphics. So far, studies that incorporate only one design feature (e.g., color; Heidig et al., 2015) and those applying various, potentially more intensive manipulations such as sounds, animations, and anthropomorphisms (Uzun and Yıldırım, 2018; Bülbül and Abdullah, 2021) are summarized under the umbrella term emotional design. Furthermore, the different ways in which the learning material is presented (e.g., paper-based or computer-based) must be considered. Previous studies were mainly computer-based (Um et al., 2012; Münchow et al., 2017; Münchow and Bannert, 2019; Stárková et al., 2019; Shangguan et al., 2020). Another aspect which needs to be considered is whether the learning material is animated or static (nature of presentation; Wong and Adesope, 2020). A meta-analysis by Berney and Bétrancourt (2016) revealed that animated graphics were more beneficial for learning compared to static graphics. Since our study was conducted in an authentic learning environment, the paper-based study books could not include sounds or animations. The resulting emotional design might, therefore, be “weaker” compared to computer-based (animated) designs [see the emotional designs applied in Uzun and Yıldırım (2018) or Bülbül and Abdullah (2021)]. Another aspect could be the learning topic. In our study we used the topic of the lake ecosystem because it is well suited for the use of concept maps. However, the effect of emotional design manipulations could vary depending on the topic, which still needs to be investigated further.

Furthermore, although in the emotional design research a direct effect of emotions on learning is assumed (Um et al., 2012), there is evidence to question this direct connection (e.g., Linnenbrink, 2006). In fact, a large body of research supports the assumption that emotions influence learner’s self-regulation, interest, intrinsic motivation, and their use of learning strategies and thereby “exerting positive effects on overall performance” (Pekrun, 2017, p. 151). To further illustrate the complexity of the relationship between learning and emotions, it has to be mentioned that it is assumed that not only emotions can influence for example motivation and achievement but they in turn influence emotions (reciprocal influence; Pekrun, 2017).

As stated earlier, the cognitive theory of multimedia learning (Mayer, 2001), presumes that learning material providing both, pictures and text, should be more efficient for learning than material consisting only of pictures or only of texts. Therefore, concept maps with picture-text combinations (emotional design and neutral design) should have been more effective for learning than text-based concept maps (control design) which we could not observe in our study neither.

Cognitive Load

We expected emotional design manipulations to influence different types of perceived cognitive load. Other than expected, emotional design did not lead to significant differences in the perceived extraneous cognitive load compared to the non-emotional designs. This finding is in line with the results of previous studies (Plass et al., 2014; Park et al., 2015). Other than the emotional design, we found the neutral design leading to a significantly reduced extraneous cognitive load compared to the control design. Emotional design features could impose more extraneous cognitive load than neutral design features since emotional designs come with additional seductive details, whereas neutral designs are – in the most literal sense – reduced to essentials. The findings support assumptions posed by the cognitive theory of multimedia learning and the emotions as extraneous cognitive load hypothesis but contradict the cognitive-affective theory of learning with media (Um et al., 2012).

Intrinsic cognitive load was presumed to be unaffected by design manipulations. Other than expected, the intrinsic load was reportedly lower in the neutral design condition compared to the emotional design and the control design condition. One potential source of intrinsic cognitive load can be lack of prior knowledge (Sweller, 2005). However, as the experimental conditions did not differ in terms of prior knowledge or the self-assessed handling of concept maps (see preliminary results, experiment 1), differently perceived complexity cannot be due to different degrees of familiarity with concept maps or different levels of prior knowledge. A possible explanation could be that learners were unable to distinguish between sources of extraneous cognitive load and sources of intrinsic cognitive load (Klepsch and Seufert, 2020). As students perceived lower levels of extraneous load, the intrinsic load of the learning material is felt less complex because more cognitive resources are available to cope with it (Klepsch and Seufert, 2020). For the settings where intrinsic cognitive load and extraneous cognitive load may merge, Klepsch et al. (2017) recommend using complex instruments to uncover underlying processes. It is also worth noting that the scale for intrinsic load had limited reliability in our experiment (α = 0.51), consequently the results need to be interpreted with caution.

For germane cognitive load, we expected the emotional design to increase the perceived germane load compared to the other conditions. Contrary to these expectations, our results do not indicate such an effect. This is in line with the findings of several studies (Um et al., 2012; Mayer and Estrella, 2014; Park et al., 2015; Navratil et al., 2018; Uzun and Yıldırım, 2018). An explanation why emotional design does not lead to higher germane load invested in a task is offered by Uzun and Yıldırım (2018). They draw the conclusion, based on Pintrich et al. (2000) that emotional design elements could give the impression that the learning material is less difficult, resulting in less effort devoted to the task. Supporting this, the students’ ratings of the perceived task difficulty in our study were significantly lower in the emotional design condition, compared to the control condition (Le et al., 2018).

Finally, methodological differences make it difficult to interpret the cognitive load results within the group of emotional design research. The self-rating questionnaire created by Paas (1992) has predominantly been used in emotional design research to measure mental effort. While, in some studies, mental effort is regarded as germane load (Um et al., 2012), in others it is equated with intrinsic load (DeLeeuw and Mayer, 2008). The same applies to task difficulty. Um et al. (2012) use this to measure extraneous load, whereas DeLeeuw and Mayer (2008) assign it to germane load. Thus, direct comparison of the results in emotional design research is difficult.

Affective State

In contrast to the cognitive-affective theory of learning with media and respective previous findings (e.g., Um et al., 2012), we did not find emotional design to induce a positive affective state (see section “Experiment 2”). Furthermore, no difference was found for other affective variables like self-report ratings on how well the students enjoyed the learning phase, desired additional lessons of a similar nature, or found the learning material appealing (see section “Experiment 1”). These results are in line with findings by many other studies which also found no influence of the emotional design on positive affect (e.g., Heidig et al., 2015; Park et al., 2015; Münchow and Bannert, 2019).

However, both the neutral and the emotional design led to lower negative affect compared to the control design. Studying concept maps can be overwhelming for learners (map shock; Dansereau et al., 1994). The excessive demand could lead to an affective reaction characterized by frustration and results according to Blankenship and Dansereau (2000) in a loss of motivation. The findings of the present study suggest that adding (emotional or neutral) images to concept maps could help learners overcome negative feelings while studying concept maps.

According to the cognitive-affective theory of learning with media, learning is influenced by emotions. Thus, it is reasonable to assume that an emotional design that does not induce emotions may also be unable to influence learning performance. Stark et al. (2018) raised the question whether changes in the emotional state of learners need to be proven when classifying an applied design as emotional. However, they note that if a change in an emotional state is required for a design to be designated as emotional design, many studies that have been carried out so far could no longer be assigned to this research field. Heidig et al. (2015) and Münchow and Bannert (2019), for example, found that emotional design influenced learning performance even though their design did not induce positive affect. Stárková et al. (2019) suspected that the affective influence of emotional design is generally small and difficult to detect [supported by the meta-analysis by Brom et al. (2018)]. It must be noted that even replication studies using the same learning material and instruments yielded contradicting results for affect (Plass et al., 2014; Park et al., 2015; Navratil et al., 2018). This suggests that the perceived affective state and learning performance could be influenced by other variables which still need to be identified.

Münchow and Bannert (2019) for example observed a time-dependent effect, whereby, after 10 min of learning time, higher (albeit not statistically significant) positive emotions were found among students who learned with emotional design material compared to students who learned with a neutral design. After another 10 min of learning, this was no longer observed. The authors concluded that emotional design, at least for the first 10 min of learning, can prevent the reduction of positive emotions. In support of this observation, many of the studies that found a positive effect on emotional state, were based on a short learning time (≤15 min). Consequently, extended learning time (e.g., 20 min, as in our study) could cause these effects to fade. Wong and Adesope (2020) support these findings in their meta-analysis and suspect that “novelty of learning with emotional designs was most beneficial between short and moderate lengths of instructional time” (p. 24).

Limitations

When measuring affective parameters using the self-reporting method, study participants may not be conscious of their current emotional state (Damasio, 2000), and even if they are, they may encounter difficulties assigning it to specific categories (Salovey and Mayer, 1990). Since our study was carried out on junior high school students rather than college students (as in most studies), the students in this study might had problems answering the items on the PANAS scale. One possible reason for the failure to find any effects on for example, appeal, enjoyment or perceived cognitive load is that the concept maps provided in the form of study books may have been more elaborate than everyday worksheet-based learning material, and as the students only learned with one of the three designs, they lacked a direct comparison. Supporting this, Navratil et al. (2018) showed that an intended negative emotional design was only perceived as more negative when students had the opportunity to compare it directly with different designs.

Conclusion and Outlook

Our research pioneered to investigate the question of whether emotional design in paper-based concept maps improve learning for junior high school students. Together, these results indicate no superiority of emotional design over non-emotional design concept maps. However, the use of emotional design seems to reduce the perceived task difficulty. The current state of research suggests that in the best-case scenario, emotions are triggered by an emotional design and learning is improved while, in the worst-case scenario, it has no effect on emotions and learning. As suggested by several authors, we agree that the effect of emotional design features could depend on factors like learning context, intensity of the design manipulation, the age of the learners or in which way the learning material is presented (e.g., Münchow and Bannert, 2019; Stárková et al., 2019; Wong and Adesope, 2020). The present study was conducted for a biological topic (as most studies in this field). Therefore caution is needed when interpreting the results, and more research is required to transfer them to other cognitive domains or different learning conditions. To allow an assessment of the generalizability of our findings, we recommend broad and systematic examinations on emotional design in concept maps for other (biological) topics.

Furthermore, to make specific statements about the use of emotional design in concept maps, more basic research on emotional design manipulations is needed. From our point of view, this research should involve the following aspects.

Reliable measuring instruments for determining emotional influence should be established. We recommend more objective approaches (Brom et al., 2018), for example, heart rate variability or electrodermal activity measurements as implemented by, e.g., Lenski and Großschedl (submitted)². Furthermore, the two dimensions of emotions (arousal and valence) should be examined differentially [Lenski and Großschedl, submitted (see footnote text 2)] as it is suspected that the learning performance is especially affected by arousal (Schneider et al., 2019; Irrazabal and Burin, 2021). Additionally, further research is needed to identify the conditions in which emotional design manipulations successfully evoke emotions (see also Heidig et al., 2015). If the basis for objective measurement and emotional induction is established, effects of those emotional design features on learning performance and emotions can be analyzed and new insights gained. These insights could have a major impact on the design of learning material and learning success in general. Since learning success could be influenced by other factors, such as time spent with the learning material, personal characteristics like cultural background (Stárková et al., 2019) or different perceptions of aesthetics (Heidig et al., 2015), uncovering these aspects should be focused on. As already stated, a large proportion of studies assume that emotions affect learning indirectly, which should also be reflected in emotional design research. Finally, the practicability of successful emotional design manipulations should be evaluated. If time-consuming and artistic manipulations are necessary to evoke emotions and influence educational variables, the use in everyday school life may be restricted [Lenski and Großschedl, submitted (see footnote text 2)].

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.

Author Contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work, and approved it for publication.

Funding

This study is part of the project “Learning Biology through Concept Mapping: Importance of a learning strategy training for cognitive load, cognitive processes, and learning performance,” which was funded by the German Research Foundation (Deutsche Forschungsgemeinschaft; DFG), Grant (GR 4763/2).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2022.807627/full#supplementary-material

Footnotes

1. ^ Lenski, S., Elsner, S., and Großschedl, J. (in preparation). Comparing Construction and Study of Concept Maps – An Intervention Study on Cognitive, Metacognitive and Emotional Effects of Training and Learning.
2. ^ Lenski, S., and Großschedl (submitted). Emotional design pictures –pleasant but too weak to evoke arousal and attract attention? BMC Psychol.

References