Understanding design-based learning context and the associated emotional experience

Abstract

Educational settings are emotional places where students experience diverse emotions in relation to academic activities and their outcomes. Emotions, in turn, greatly influence students’ learning process and engagement. Research on emotions in Design-Based Learning (DBL) has so far been coarse-grained examining how students evaluate their overall feelings towards the DBL project. As yet, little is known regarding how specific DBL activities influence students’ emotional experience. Therefore, we report a three-month field study of a DBL project involving 30 middle school students (aged 13–14) addressing dual research purposes: (1) to faithfully reconstruct the manner and sequence of activities during DBL from a fine-grained perspective; and (2) to examine the relationship between these activities and students’ emotional experience. This study used a mixed research method and collected multiple data sources, including experience sampling surveys, classroom observations, and interviews. The research outcomes in this study are multiple. First, this paper reveals detailed inspection regarding the types of task students performed, the strategies of shifting, and executing tasks during the process student experienced. Second, this paper identifies specific types of activities that have a significant positive or negative relationship with students’ emotional experiences. Derived from reported empirical evidence in the present study, this paper furtherly proposes an Activity-and-Affect model of DBL. This model provides a fine-grained description of DBL activity as continuous along three dimensions: task (design thinking process, project management, social interaction) task strategy (single-tasking vs. multitasking), and collaboration strategy (individual or group). Our analysis highlights the variability in how different DBL activities can be associated with different emotions.

Introduction

Design can be seen as a valid form of inquiry, given the general goal of design activities is typically to develop or improve artefacts and services (de Vries 2016). Consequently, the design process can be easily transferred to a process of problem-solving in the learning of many subjects in the K-12 classroom (Davis et al. 1997). Driven by the mechanism of design, the learning approach so-called Design-Based Learning applies Design Thinking (Rowe 1987) in a problem-based or project-based learning context. Design thinking is built upon fundamental mindsets (e.g., human-centeredness, empathy, a culture of prototyping, and radical collaboration) that frame a student’s orientation to learning (Ford 2010). The application of design thinking in education allows students to develop designerly ways of doing and knowing (Cross 1982) from moving back and forth of a sequence of design phases (e.g., insights, investigation, ideation, and implementation). In this way, students experience and acquire the concept and knowledge presented in the design project. Related research has provided ample evidence of the potential benefits of DBL: gain in self-confidence (Hendricks and Fasse 2012), become more tolerant of errors (Martin 2015), and increase students’ interest in the subject topic (Reynolds et al. 2009). Consequently, DBL as a learning approach is attracting increasing attention by relevant communities, e.g., for Science, Technology, Engineering and Mathematics (STEM) education (Apedoe and Schunn 2013; Davis 1998; Doppelt et al. 2008; Fortus et al. 2004; Kolodner et al. 2001; Mehalik et al. 2008; Nelson 2004) and design education (Davis et al. 1997).

One of the featured characteristics of DBL is engaging students in complex and sometimes interleaved processes of inquiry and designing. Specifically, the inquiry process in DBL forces students to carry out the design process in a flexible and highly opportunistic manner (Razzouk and Shute 2012), which involves frequently switching between different tasks and sub-problems. As a result, students often are expected to go through the design process iteratively rather than adhere to some strictly defined stepwise process (Dym et al. 2005). The opportunities for learning in DBL may precisely occur during this active interleaving and switching between different roles in individual work or teamwork, and between various sub-tasks amidst the process. For instance, interleaving sub-tasks such as investigative activities with attempts to take on a design challenge is expected to enable students to refine their understanding of concepts, to ideate solutions to design problems, to communicate and to collaborate all at the same time (Fasse and Kolodner 2000). However, such a complex and dynamic process makes positioning or orchestrating specific activities within the overall DBL process challenging for researchers and practitioners. Perhaps, for this reason, there has been little rigorous empirical analysis of how DBL activities unfold and the emotions students experienced during these.

Research has shown that emotions can influence students’ attention, engagement, and their choice of learning strategies (Efklides and Volet 2005; Pekrun 2014; Skinner et al. 2014). It has been argued already that emotions are essential in DBL (Zhang et al. 2018; Zhang et al. 2020). However, research on young students’ emotions in DBL has so far been coarse-grained (Zhang et al. 2020). For example, emotion research in DBL may pertain to how students evaluate their interest in the subjects covered (Hendricks and Fasse 2012; Karahoca et al. 2011), or their willingness to participate in similar DBL activities in the future (Buechley et al. 2008; Giannakos et al. 2014). Existing research seems not yet to provide a fine-grained consideration of how emotions are associated with specific DBL activities.

To address this apparent gap in related research, we carried out a three-month field study in a DBL classroom with 30 middle school students (aged 13–14). Overall, the dual aims of this study are: (1) to develop a fine-grained understanding of how DBL activities unfold in the classroom, and (2) to examine students’ associated emotional experience during these. To reconstruct the dynamic processes of DBL activity, we refine DBL activity from several aspects such as learning activity, grouping strategy, social interaction, and temporal of the learning task. To conceptualize emotional experience, we refer to eight representative achievement emotions (as seen in Table 1) and two aspects of engagement (in Table 2) in the present study. These rationales are furtherly detailed in the Background section.

Table 1 The achievement emotions in the present study

Table 2 Engagement aspects of emotion experience

Specifically, the present study addresses the following two central research questions:

RQ1

How do DBL activities unfold during the process? (e.g., in which manners and sequence DBL activities reveal by students?)

RQ2

What relationships exist with students’ emotional experience, when engaged in different DBL activities? (e.g., which DBL activities have a positive and which have a negative association with students’ emotional experience?)

This paper makes the following contributions to the field: (1) an empirical understanding on how students go through the DBL process including the types of task students performed, the strategies of shifting and executing tasks during the process student experienced; (2) developing an understanding of the relationships between students’ emotional experience and different DBL activities; (3) a theoretical model which highlights the complex and reciprocal relationships between DBL activities and associated emotional experience; and (4) implications for informing further practice and research for carrying out DBL activities.

Background

Design-based learning

Learning can be addressed from a variety of perspectives. A constructivist learning perspective asserts learning is affected by the context, namely, in which an idea is taught, as well as by the students’ beliefs and attitudes (Olusegun 2015). In line with the constructivist learning tradition, DBL has been introduced as a learning approach in which the typical set up has students encountering a design challenge and attempting a solution individually and/or in small groups using only prior knowledge (Kolodner et al. 2001). Other definitions of DBL are quite comparable, e.g. (Gómez Puente et al. 2013a, b; Mehalik et al. 2008; Nelson 2004). In general, DBL is akin to other terms, such as Learning-by-Design [LBD; e.g., (Kolodner et al. 2001)], Learning through Design (Resnick and Ocko 1990), Design-based Science [DBS; e.g., (Fortus et al. 2004, 2005; Vattam and Kolodner 2006)], T/E Design-based Learning [T/E DBL; e.g., (Shanta and Wells 2020; Wells 2017)] or maker education [learning by making; e.g., (Blikstein 2013; Chu et al. 2016; Martin 2015)].

Prior attempts in relation to characterizing DBL activities fall into the following two categories: (1) The requisite phases of the design process and/or related inherent knowledge construction; Related works include, such as the conceptualization of a five-stage design thinking process (Stanford d. School. 2013), the learning by design cycle (Kolodner et al. 2001), the design-based science learning cycle (Fortus et al. 2005), the activity model for the process of scientific inquiry (Harwood 2004), and PIRPOSAL phases of integrative STEM education (Wells 2016). These works illustrate the designing and inquiry processes through stages of constructing design concepts and alternatives. (2) The critical instructional elements for implementing a successful DBL curriculum in practice; For instance, some studies have pointed out the iterative design process as a vital element in orchestrating DBL activity (Adams and Atman 2000; Bekker et al. 2015; Gómez Puente et al. 2013a, b; Kolodner 2002; Kolodner et al. 2003; Lewis et al. 2018). Other elements include the teacher’s role of coach (Gómez Puente et al. 2013a, b; van Diggelen et al. 2019), reflection (Bekker et al. 2015; Jun et al. 2017), the open-ended and authentic project feature (Bekker et al. 2015; Gómez Puente et al. 2013a, b), and collaboration in teamwork (Bekker et al. 2015; Fortus et al. 2004). These works help develop successful instructional strategies for a DBL environment and contribute to a theoretically grounded understanding of DBL. Literature in the second category characterizes DBL in a relatively broader sense from an instructional design perspective, while in the first category addresses the design and inquiry nature of DBL that has been widely agreed as the core of such a learning approach.

However, given the dynamic practices of designing and inquiry processes in DBL, current studies will not suffice in providing detailed accounts of how students approach and unfold tasks amidst DBL. For instance, they do not consider in detail how DBL unfolds in action from a student-centred perspective, and whether or how students interleave the aforementioned prescribed DBL elements while the learning activity takes place. In this paper, we refer to these mentioned critical aspects, especially regarding the designing and inquiry process of DBL, to reconstruct how DBL activities can be unfolded and approached by students; and to understand how DBL activities can be conceptualized and represented.

To understand the DBL activity at a systemic level, we adopted the curriculum development components (van den Akker et al. 2010) as a theoretical foundation. This curriculum development framework proposed by van den Akker et al. (2010) includes ten components: learning activity (i.e., how are students learning?), teacher role (i.e., how is the teacher facilitating their learning?), grouping (i.e., with whom are students learning?), time (i.e., when are students learning?), content (i.e., what are students learning?), location (i.e., where are students learning), aims and objective (i.e., what learning goals do student pursue?), assessment (i.e., how is their learning assessed), materials and resources (i.e., with what are students learning) and rationale (i.e., why are students learning). Theoretically, learning in DBL may take place in a student’s construction of the object and interaction with the people around, according to the notion of “constructionism” (Papert and Harel 1991). Therefore, we mainly refer to some components of the curriculum development framework (van den Akker et al. 2010), which helps construct the student’s dynamic engagement and interaction with the tasks and people during DBL. In this paper, we refine the DBL activity as situational information on four interdependent aspects.

• The learning activity (e.g., the sequence and ritual of design and learning process: how tasks are unfolded and how students are learning);

• The grouping strategy (e.g., the allocation of learning tasks: student learning individually or in small groups);

• The social interaction (e.g., peer-to-peer communication and student–teacher interaction);

• The temporal aspects of the learning task (e.g., the duration on and sequence of learning tasks).

For example, one possible DBL instance could be described along these dimensions as follows: a student works individually (grouping strategy aspect) on interleaving between ideating design concepts and empathizing with design users (learning activity aspect), and the teacher provides support to this student (social interaction aspect) when this student is interleaving between these two tasks for a long time (temporal aspect).

DBL, emotion, and engagement: an extended perspective

Emotional experiences are ubiquitous and greatly influence the learning process. Some researchers consider emotional experiences as a “way of being” and study them as holistic episodes (Schutz et al. 2006). Others argue that the emotional experience is characterized by at least two levels of consciousness (Frijda 2005; Lambie and Marcel 2002). The first order of emotional experience is characterized by total involvement (Schutz et al. 2006), e.g., being in a state of “flow” in tasks or immersed a feeling. The second-order adds awareness (Frijda 2005; Lambie and Marcel 2002), which adds specificity and direction of attention (Schutz et al. 2006). For instance, being conscious of the situation where the potential for emotional regulation during the experience occurs. A third-order refers to the quality of emotional experience (Frijda 2005), which can be thought of as a motivational tendency that helps to define how emotional expression should look and feel in a particular situation (Schutz et al. 2006).

In the field of DBL, there has been increasing attention placed on studying or measuring emotion as a part of the evaluation of DBL activities (Barak and Doppelt 1999; Hendricks and Fasse 2012; Neve and Keith-Marsoun 2017; Reynolds et al. 2009; Tisza et al. 2019; Zhang et al. 2018). This body of emotion research in the DBL field often treats emotions as an independent outcome indicator for the students’ experience of a DBL intervention, becoming separated from the holistic multi-faceted nature of the emotional experience itself. The present study used eight representative achievement emotions (as seen in Table 1, reflecting on the first-order of involvement) and two aspects of engagement (in Table 2, reflecting on the second-order of awareness) in framing emotional experience. The general conceptualization of emotional experience in DBL is further detailed in the following sections.

Achievement emotion

In emotion research, some models and theories use a limited set of categories to classify emotions (Feidakis et al. 2014), echoing the idea that emotion is information (Calvo et al. 2009). The Control-Value theory (Pekrun et al. 2007) defines achievement emotions as emotions tied directly to achievement activities or achievement outcomes. More specifically, this theory treats achievement emotions as three-dimensional constructs (Pekrun et al. 2007) defined along the dimensions of object focus (i.e., activity-focus vs. outcome-focus), valence (i.e., positive vs. negative) and activation (i.e., activating vs. deactivating). Accordingly, activity-focus emotions can be either positive activating (e.g., enjoyment), positive deactivating (e.g., relaxation), negative activating (e.g., anger, frustration), or negative deactivating (e.g., boredom). Likewise, outcome-focus emotions can be positive activating (e.g., joy, hope, pride, and gratitude), positive deactivating (e.g. contentment, relief), negative activating (e.g. anxiety, shame, and anger) and negative deactivating (e.g. sadness, disappointment, and hopelessness).

Few studies have directly examined achievement emotions in different learning contexts. For instance, one study (Lichtenfeld et al. 2012) assessed elementary school students’ enjoyment, anxiety, and boredom about the situations of attending class, doing homework, and taking tests and exams. Another study (Pekrun et al. 2011) examined enjoyment, anxiety, boredom, and six other emotions (including hope, pride, relief, anger, shame, and hopelessness) in these similar situations but with university students. More recently, a study (Vongkulluksn et al. 2018) examined frustration, confusion, excitement, and curiosity, specifically during an elementary school DBL course. All these studies represent the different range of contexts for studying achievement emotions. However, the achievement emotions examined in these studies differ. The study by Pekrun et al. (2011) with university students, covers all three dimensions of achievement emotions resulting in broader coverage of achievement emotions than the other two studies involving elementary school students (Lichtenfeld et al. 2012; Vongkulluksn et al. 2018) which only address one or two dimensions. However, that study only considers emotions at a macroscopic level without making precise the relation of emotions to specific DBL activities.

As argued in a study conducted by Linnenbrink-Garcia and Pekrun (2011), more research is needed to consider how different contexts shape students’ emotions. The present study focuses on a wide variety of students’ achievement emotions, especially concerning a DBL course in a middle school. The focus of achievement emotions in the present study is elucidated in Table 1, including eight representative achievement emotions, to ensure a more balanced coverage compared with the studies mentioned above. There are two considerations for selecting achievement emotions in the present study. First, the emotions chosen are according to the three-dimensional taxonomy in an equally representative way. Second, the emotions chosen are echoing the ones that have been addressed with K-12 students, e.g., pride, enjoyment, relaxation, boredom, anxiety, and hopelessness in prior studies (Carroll et al. 2010; Giannakos et al. 2014; Giannakos and Jaccheri 2013; Zhang et al. 2018).

Engagement

Students’ engagement that reflects psychological investment and effort is necessary for learning (Shernoff 2013; Zollars 2017) and highly associated with emotions. Related works on the definition of engagement have different foci. For instance, the Flow theory (Csikszentmihalyi 1997) conceptualizes student engagement based on the culmination of concentration, interest, and enjoyment (i.e., flow). Besides, the Broaden-and-Build theory (Fredrickson 2004) hypothesized that positive emotions broaden one’s thoughts and actions, defining student engagement as a multidimensional construct including academic, behavioral, cognitive and psychological subtypes, and considering engagement as conducive to meaningful academic and social-emotional outcomes (Fredricks et al. 2004).

The notion of engagement in the DBL has been addressed using different theoretical perspectives. For instance, one recent study in a maker space (Zollars 2017) considers engagement as a component of flow, while another study (Guo et al. 2017) has measured the influence of gender on students’ cognitive, behavioral and emotional engagement in a high school DBL course. Furthermore, one study (Doppelt et al. 2008) focused on one aspect of engagement, which, to some extent, amounts to students’ participation to analyze the close association between engagement and achievement. In summary, a consistent manner of assessing engagement seems lacking in the body of DBL research.

Therefore, we put forward our conceptualization of engagement in the present study. This conceptualization is built on an integrated understanding of engagement according to the Flow theory and Broaden-and-Build theory. Table 2 illustrates this engagement conception, which includes the degree of concentration (i.e., how well you are concentrating?) and its perceived associated outcomes (i.e., were you learning anything or getting better at something?).

Method

We report on a three-month case study of a class with 30 middle school students engaging in DBL activities carried out as part of the standard Dutch school curriculum in design and research. The study had a dual purpose of studying the activities in the DBL context and the students’ associated emotions experience. A triangulated mixed-methods approach (Creswell 2015) was adopted to collect both quantitative and qualitative data regarding the emotions experienced during the DBL activities, using a combination of naturalistic observation, interview, and experience sampling survey.

Study context

The present study was embedded in a Design & Research subject (i.e., O&O; the Dutch abbreviation for “Onderzoeken en Ontwerpen”) at a Dutch public school so-called “Technasium.” Students at Technasium develop STEM-related competences and design skills (e.g., creativity, collaboration, communication, planning, and project-based work). Overall, the Design & Research (O&O) subject in Technasium education is closely related to STEM contexts (Vossen et al. 2019), in which typically combines science content-related research projects (e.g., DBS (Fortus et al. 2004) or LBD (Kolodner et al. 2001)) and technological and engineering-related design projects [e.g., T/E DBL; (Shanta and Wells 2020)]. For example, the Research & Design projects can be done with different STEM-related contexts, such as relating to architecture, industrial design, or biology (Vossen et al. 2019).

In the present study, the design challenge of this DBL project was to design an escape room for the local fire department to allow participants to playfully and interactively improve awareness of fire safety in and around the house. In this Escape Room project in the present study, the design challenge students have to address is more related to the technological and engineering content and knowledge about design and inquiry. The learning goal in this Escape Room design project is open-ended and depending on the student-oriented personal learning objective. In general, the typical learning goal in this project is to learn design and inquiry skills (e.g., empathize with the client’s need, inventing, modeling, use, and value information) and collaboration skills (e.g., communicate and cooperate with peer students and stakeholders). The design brief of this Escape room project is described in detail in “Appendix 1”. The learning outcomes of this escape room DBL project includes students’ puzzle design artefacts (see examples in “Appendix 2”) and their project reports documenting how they develop their design works.

Before the study was initiated, we ascertained that the planned learning activities match four DBL characteristics that have been used in prior studies (Zhang et al. 2018; Zhang et al. 2020) to ensure external validity. These following four DBL characteristics are built upon the features of DBL that have been addressed in the literature, e.g., (Bekker et al. 2015; Chen and Chiu 2016; Davis et al. 1997; Doppelt 2009; Doppelt et al. 2008; Gerber et al. 2012; Gómez Puente et al. 2013a, b; Kim et al. 2015; Kolodner et al. 2001).

• (a) The learning activity should be open-ended, followed by a design process, and involving multidisciplinary knowledge and skills. Furthermore, the challenge and problem introduced in this learning activity should be embedded in an authentic context.

• (b) The teacher’s role should be that of a coach enabling the students to become active learners.

• (c) The grouping should enable students to share a sense of responsibility and be able to communicate and collaborate with peers and stakeholders.

• (d) Materials and resources should support hands-on activities, e.g., prototyping or testing and minds-on activities, e.g., empathizing, defining, or ideating.

Participants

Participants were a class of 30 middle school students aged 13–14 who are at the level of HAVO2 (equivalent to the grade 7–8 in a secondary education system). In the Dutch education system, the HAVO stream gives access to polytechnic level tertiary education, which in the Netherlands is considered an applied science university. This stream is not the most challenging in secondary education, but one where science and design skills are of high importance. Participants in this study already had 1 year of experience with STEM and DBL. Before participating in the Escape room design project, they have already been involved in two design projects (e.g., designing a working toy for pigs in an organic form and designing a system encouraging students to classify garbages). Besides, they have also participated in several skill training modules, such as technical drawing and sketching, making video presentations, and working with workspace tools and materials (e.g., wood, acrylic, cardboard, etc.). The guardians of participating students provided informed consent before the start of the study.

Measures

To overcome the challenges involved in obtaining reliable data about young students’ emotions in the DBL classroom (Zhang et al. 2018), e.g., a social-desirability bias for self-reporting internal states, data collection in this study involved multiple sources of information, including an experience sampling survey—EmoForm (Zhang et al. 2019), observation, and interview. The data collection was carried out by two of the authors who conducted an after-project interview and two research assistants (students in a design department with more than two-year experience in DBL in higher education) who attended field observations and conducted after-lesson interviews during the project.

Experience sampling survey—EmoForm

EmoForm (Zhang et al. 2019) is a retrospective experience-sampling questionnaire (see Fig. 1) for use in the constructive or creative learning context that combines negative and positive items. An earlier validation study has shown EmoForm has an acceptable internal consistency and is a helpful self-report tool for capturing students’ emotional experience in DBL (Zhang et al. 2019).

Fig. 1

Example of a filled EmoForm (a made-up example based on actual reports)

The content of EmoForm consists of four sections of questions: (1) what activities were a student involved in sequence in the past lesson? (2) How a student felt (precisely about four activity-focused achievement emotion, e.g., enjoyment, relaxation, boredom, and frustration) every 25 mins during the last lesson? (3) How well were a student concentrating and how much did a student learn in every 25 mins in the previous lesson?; (4) How a student felt about four outcome-focused achievement emotions, e.g., contentment, hopelessness, anxiety, and pride in the past lesson? In the present study, students were required to fill in the EmoForm at the end of each DBL daily based lesson.

Observation

Combined with students’ self-reporting with EmoForm, we used an observation note sheet which applies continuous recording (Miltenberger 2012) where observers record: (a) instances of the episodes following our recommended structure (i.e., activity events preceding, following emotions, following engagement level or behaviors); (b) the episodes of three to four students who are in the same team; (c) the episodes in every 25 mins (the same as the timeframe in EmoForm) which allows triangulating self-reporting responses and observation notes within the same period.

We’ve involved two participating observers (i.e., research assistants) who were trained before the start of the study. They observed one team using the observation note during every lesson. Every group in the class is observed in a pre-determined order over time.

Interviews

Two types of interview were conducted during and after the DBL project. (a) a semi-structured interview following a predetermined protocol was conducted during the project. Specifically, one observer observed a specific team (following a predetermined order) at every lesson. Before the daily lesson ended, the observer interviews the same group of students one by one. The questions during the interview revolved around one or two episodes the observer has recorded. (b) After the end of this project, a structured in-depth interview was conducted with the students who have reported multi-tasking activities, to get a deeper understanding of reported cases.

Procedure

Seventeen lessons were scheduled during this DBL course lasting three months. The lesson duration varied: three sessions of 50 mins, eight sessions of 100 mins, and six sessions of 150 mins. As shown in Table 3, all seventeen lessons/sessions were observed using our observation note sheets. However, semi-structured after-lesson interviews and EmoForm were completed only in fourteen and fifteen sessions, respectively. The exceptions, where data was not collected from both measurements, concern some unexpected situations (a canceled lesson, and a visit to the local fire department where it was not possible to distribute questionnaires nor interview). Besides, one structured in-depth group interview was conducted after the end of this project.

Table 3 The procedure of scheduled lessons and data collection

Analyses

In this paper, we analysed the data collected from EmoForm as the primary source, while observations and interview recordings as supplementary sources. The quantitative analyses are made with IBM SPSS statistics 25 (IBM Corporation 2011).

To answer RQ1 (How do the DBL process unfold?), we first qualitatively analyzed the data collected by observations and EmoFoms following a thematic analysis approach (Braun and Clarke 2006). We transcribed EmoForm recordings and observation notes following the predefined episode structure and inductively coded the transcripts into categories. For instance, we first transcribed and marked all newly emerged activities on EmoForm recordings (e.g., off-tasking events, design documentation, and planning, etc.) and observation notes (e.g., off-tasking activities, asking and/or offering help, reviewing other’s tasks, and chatting with peers). We then performed a follow-up thematic analysis to find the featured categories of all reported DBL tasks (as seen in Table 4). Detailed results in terms of DBL tasks can be found in the task dimension Result section.

Table 4 The categorized activities in the task dimension

Additionally, we descriptively reported sequences of switching tasks based on data from both EmoForm and observation notes, and we portrayed the overall evolution of participating DBL tasks per student over days (as seen in Fig. 2). Likewise, we reported the manner of combining tasks based on EmoForm data, and we then furtherly validated these results through interviews. More detailed results can be found in the task strategy (single-tasking vs. multi-tasking) Result section. We also analysed the highlighted types of combined tasks based on the frequency of reported students (as shown in Table 5), and we portrayed the relationship between combined tasks during DBL (as seen in Fig. 3). For executing tasks in the manner of groups or individually, we analysed the types of tasks favoured in teamwork and individual work based on the reported time spent. Besides, we also calculated students’ average time spent on tasks when utilizing specific collaboration strategies (as shown in Tables 6 and 7). Detailed results can be found in the collaboration strategy (collaborative-tasking vs. individual-tasking) Result section.

Fig. 2

Evolution of participating DBL activities (per child) over days

Table 5 Highlighted task co-occurrence during multi-tasking

Fig. 3

Relationship between DBL activities in multi-tasking

Table 6 Average time (in mins.) on collaborative-tasking (CT)

Table 7 Average time (in mins.) on individual-tasking (IT)

Based on the qualitative results relating to RQ1 (How do the DBL process unfold?), we synthesized a three-dimensional perspective of DBL activities (task, task strategy, and collaboration strategy dimension). Based on these results, we furtherly proposed the Activity-and-Affect model of DBL (as seen in Fig. 4) to establish the nuanced channels between DBL activities and students’ emotional experiences. To assess the goodness of fit of the model, we fitted the data repeatedly collected by EmoForm into the taxonomy of this proposed model and then performed a linear regression using a hierarchical data structure (as shown in Table 8). Detailed analysis procedures and results can be found in the subsection of the Result section—A conceptual framework: the Activity-and-Affect model of DBL.

Fig. 4

The conceptual Activity-and-Affect model of DBL

Table 8 Coefficient regression using a hierarchical data structure

Finally, to answer RQ2 (How are emotional experiences related to different DBL activities?), we performed multiple linear regressions using a stepwise method to furtherly measure which DBL activities significantly contribute to students’ emotional experience (see Table 9 in “Appendix 3”). Specifically, this analysis investigates DBL activities from the dimensions of the task, task strategy, and collaboration strategy, respectively. All the multiple regression analyses were calculated on a group level, rather than distinguish individual differences. A detailed explanation of the analysis procedure and results are presented in the Results section.

Table 9 Coefficients of multiple linear regressions

Results

Uncover DBL activities from a three-dimensional perspective (RQ1)

We report the findings relating to RQ1 (How does the DBL process unfold?) from the following three aspects: (a) task dimension: various tasks and varied paths of the process, (b) task strategy dimension: multi-tasking versus single-tasking strategy, and (c) collaboration strategy dimension: collaborative-tasking versus individual-tasking used in the process. For each aspect, we first report the empirical findings derived from the present study and then discuss the theoretical implications relating to the literature. To the end, we propose a conceptual framework called the Activity-and-Affect model of DBL to construct a three-dimensional perspective of DBL activities and to describe a flexible view that taking account of how different DBL activities could be associated with mixed emotions.

Task dimension: various tasks and varied paths of the process

In general, investigating how the process unfolds is essential for a proper understanding of context and its influence on behavior (R. Mehl and Conne 2012). In this section, we report how the DBL process unfolds (RQ1) by illustrating the various tasks and varied paths of the process that students followed (see Fig. 2), which was derived from the data collected with the observation (left side of Fig. 2) and EmoForm (the colourful square block of Fig. 2).

On the top legend of this figure, each colour represents one type of tasks documented in EmoForm. Each square block in the table of Fig. 2 represents 1 day. The colourful stripes in the square block represent the tasks a student has completed within a day. For instance, the data described in Fig. 1 from a student, can be compressed into a single square block filled with five differentiating coloured strips (resembling the block of student H4 on day-2 in Fig. 2). The whole colourful square table describes all participated students involved activities in this DBL project. Nine teams were denoted in this figure alphabetically ranging from “A to I,” and each group (containing three to four individual students) was denoted by a number, e.g., A1, A2 A3. The left side of this figure illustrates the observation data and primarily lists types of activities that were identified by observations rather than predetermined by EmoForm.

Due to the open-end nature of DBL, students have autonomy in planning and implementing tasks. As a consequence, how they went through a sequence of DBL activities varied across teams. For example, only teams B and E experienced all predetermined task-focused activities foreseen in the design of the EmoForm (refer to the top legend in Fig. 2). It is because some teams missed reporting some activities. Indeed, we observed that some teams (e.g., team A) participated in design documentation, and others (e.g., C, D, and F) participated in the planning despite that participants did not report on this. There is also quite some variation in how teams experienced the DBL process: e.g., two teams (F and G) did not report social interactions (getting support from a teacher or giving feedback to others). Moreover, the degree of iteration in the process also varied substantially between teams. For example, team H went through activities iteratively most times, whereas teams F or G the least times.

Students seemed to follow a relatively consistent activity pattern over time. As one might expect, the activity of empathizing and understanding users (EDU) was reported mostly at the initial steps of their design process (almost all students report this on the first day and slightly fewer on the second). Continuing on the general pattern, we see that the activity of defining a design problem (DDP) very often happens together with, or right after the event of empathizing with the design user (EDU). There were more variations with regards to ideating design solutions (IDS), which took place early on or even only in the last few days of the project (Fig. 2).

Despite that all teams participated in defining the designing problem (DDP), it was only a small number of individual students who did so: on the ninth day of this project, eighteen students in total. The activity of making prototype (MP) and testing prototypes (TP) was left for the later phases of this design project. On the 13th day, all teams were involved in making and testing. The project ended uniformly for all groups and individuals as they were all engaged in preparing and presenting (PP) their work. Getting support from the teacher (GST) and getting feedback from others (GFO) happened at various times during the project with no apparent pattern, most often in parallel with other activities. Too few instances relating to design documentation (DD) and planning (PL) were recorded to allow discerning a common pattern.

Observation data shows some patterns in the learning process. Some off-task cases were indicated in classroom observations, such as dealing with a computer problem, searching for lost documents, looking at the phone while waiting for teacher’s support, and goofing around. Apart from these non-task related activities, the most commonly reported events were documenting their designs and planning. Besides, the field observations in this study also discerned another three frequent activities relating to students’ task-related social interaction amidst DBL. (a) Asking or offering help (AOH). It can be expressed by answering peer’s questions and asking for instructions on the task, or students sometimes would help a team member with a task when they were doing some other tasks. Besides, we found offering help was often concerning trying out another team’s prototype. (b) Reviewing other’s work (ROW). It often occurred through testing prototypes and when following the presentation of another group. (c) Chatting with peers (CWP). It often happened in idle moments while waiting for other tasks, such as presentation, getting the teacher’s support, and sometimes during an off-topic conversation.

In general, characterizing and categorizing DBL activities is a fundamental step to identify the most likely elements of DBL and support learning and teaching in practice. Based on the results derived from this study, we outlined a list of project-related tasks in Table 4, which comprises the three most likely categories: Design Thinking Process (DTP), Project Management (PM), and Task-related Social Interaction (TSI). The mixed steps and tasks in parallel (see the last four rows of Table 4) will be furtherly reported and discussed in the next sub-section in terms of the multitasking strategy.

Design thinking is a general theory of design (Buchanan 1992) that has been used to characterize what individual designers know and how they approach and make sense of their work (Kimbell 2011). The design thinking process category we proposed in this table is consistent with the widely adopted design process recommended by the Stanford d. school (2013), which consists of five main steps: Empathize, Define, Ideate, Make and Test.

Furthermore, this table identifies the category of project management to refer collectively to some auxiliary activities relating to the DBL context, e.g., presentation, design documentation, planning. Notably, an earlier study (Nieswandt and Mceneaney 2012) has regarded design documentation as one of the additional essential design skills in the high school classroom. In another study (Doppelt et al. 2008), presentation in DBL is the situation where teacher assessment and peer-assessment takes place. Besides, planning is regarded as a critical skill or practice for learning science (Kolodner 2002).

The final category pertains to social interaction, which includes getting support from the teacher or getting feedback from others. The importance of social context has already been emphasized in earlier works, e.g., the teacher’s supportive coach role on students’ tasks and processes in DBL (Bekker et al. 2015; Gómez Puente et al. 2013a, b). While earlier studies have argued that it is more enjoyable to do tasks with peers, e.g., see (Carroll et al. 2010), little empirical evidence of such collaboration can be found in DBL literature. Table 4 explicitly emphasizes project-related social interaction and de-emphasizes off-topic social interactions, e.g., goofing around, making jokes. Besides, task-related social interactions are further classified as asking or offering help, reviewing other’s work, and chatting with peers (e.g., casually discussing with peers).

Task strategy dimension (multi-tasking versus single-tasking strategy)

During the DBL process, students sometimes engaged in one singular task after the other, and at other times carried out multiple tasks in parallel. Single-tasking refers to a single activity (e.g., one specific stage of the design thinking process, or separate action of receiving help from the teacher, etc.) on which students dedicated a sustained period before interleaving and switching to other tasks. For instance, the EmoForm in Fig. 1 illustrates a single task from the timeframe of 50 min until 75 min. On the contrary, multi-tasking refers to two to three project-related tasks carried out in parallel within the same timeslot. Note that the term multi-tasking here does not apply to non-project-related tasks (such as working while listening to music, or goofing around, etc.).

From the data collected from the first section of EmoForm, students reported working on a single task more often rather than in multi-tasking. For single tasks, students tend to spend the most time on empathizing design users (M = 365 mins, SD = 275 mins), making a prototype (M = 328 mins, SD = 213 mins), and ideating design solutions (M = 318 mins, SD = 205 mins). Overall, students worked on at most two to three tasks within any single time interval (25 mins), and six types of task co-occurrences can be discerned that are detailed below. Notably, the first and third types of task co-occurrences stand more robust than the rest types based on the frequency of involved students.

• (1) Combining different design process steps;

• (2) Combing task-related social interaction;

• (3) Mixing design process steps and task-related social interaction;

• (4) Combining design thinking process steps and project management;

• (5) Mixing project management and task-related social interaction; and

• (6) All categorized tasks in parallel.

When interviewed about their multi-tasking behavior, most students did not mention any motivation or purpose for starting to multi-task. During the interviews, students reported multi-tasking activities when they were involved in different successive tasks (or, in other words, sequential tasking) in a single time interval (e.g., 25 mins). Furthermore, analysis of the interview data (which were also consistent with the results obtained from the observations and the EmoForm) helped identify the following two particular situations which appear to be related to multi-tasking:

• (a) When helping out a team member working on a different task than them; this was confirmed by our observation in the classroom where we could notice, e.g., one child making prototype while helping other with painting their work.

• (b) When tasks are inherently interdependent, in which case they discover new insights for one task while working on a different task [reflecting the opportunistic nature of the design process (Guindon and Raymonde 1990)]. For example, students explained that the task of making the prototype in parallel with ideating for the cases in which they would come up with new ideas (little changes or additional features) when they were building their prototype. Moreover, tasks (e.g., empathizing and ideating) were done in parallel in some cases, in which the team was searching for design inspiration. The combined task of defining the design problem and ideating design solutions happened when students discovered a new aspect of design problems during the time they were designing their solutions, which also in line with the observation findings.

Figure 3 visualizes all the co-occurrence relationships between different DBL tasks, which are extracted from EmoForm data. This figure shows how DBL activities were all be combined to some extent, except for planning. Nodes in the graph represent different types of activities, while lines in the figure represent by their thickness how often the specific activities are combined in multi-tasking. How often here means how many students have reported such a co-occurrence relationship between DBL tasks.

Overall, there is a stronger co-occurrence between empathizing design user (EDU), ideating design solution (IDS), getting support from the teacher (GST), and getting feedback from others (GFO). Additionally, the task of empathizing with the design user (EDU) was carried out throughout the entire design process and often in parallel with other minds-on activities (e.g., DDP, IDS) and also in parallel with other’s input such as teacher’s support (GST) and other’s feedback (GFO). Interestingly, identifying the design problem (DDP) and testing prototype (TP) were not combined. Defining the design problem (DDP) is often combined with empathizing with users (EDU), which is a good example of the first type of task co-occurrence (i.e., connecting different design process steps) as described in the bullet lists above. This combination is understandable, as it provides a springboard for an in-depth understanding of the design challenge. Interestingly, as an example of the fourth type of task co-occurrence (i.e., combining design thinking process steps and project management), one child reported combining the activity of defining the design problem (DDP) with design documentation (DD).

The open-ended nature of DBL, on the one hand, gives students freedom in task implementation to encourage diversity in design approaches (Gómez Puente et al. 2013a, b). On the other hand, the opportunistic nature of the inquiry process inherited from the design thinking notion encourages students to move among tasks (Razzouk and Shute 2012). Such an opportunistic approach is well known and may include, for example, immediate recognition of a partial solution in another part of the problem, immediate handling of inferred or added requirements, drifting through partial solutions, and interleaving problem specification with solution development (Guindon and Raymonde 1990). This may be the reason why students combine different DBL tasks to varying extents. To theoretically describe this manner of executing tasks in DBL, we introduce the task strategy dimension includes a dichotomy of strategies—Single-tasking (ST) and Multi-tasking (MT). The distinction of multi-tasking from single-tasking in this paper pertains to how and whether tasks are inherently interdependent or may reflect an opportunistic approach to solution development. Specifically, we refine this task strategy dimension of DBL activity as follow:

Single-tasking refers to students spending a continuous time interval on a single task before interleaving and/or switching to others. For example, as seen in Fig. 1, the student applied the single-tasking on ideation between minute 50 and minute 75.

Multi-tasking is defined by Pashler (1994) as a mode of doing multiple activities simultaneously in an interleaved manner. Defined by Salvucci et al. (2009), multi-tasking is represented along a continuum in terms of the time spent on one task before switching to another. However, the emphasis of multi-tasking representing such phenomenon was largely lacking and underrepresented in prior DBL works. Multi-tasking in this paper refers to performing two to three project-related tasks contrary to how multi-tasking is often defined in earlier works, i.e., including off-topic tasks done either concurrently or sequentially in a particular time interval. For example, in the case of Fig. 1, the student applies multi-tasking on empathizing with the design user and getting support from the teacher during the first 25 mins.

Collaboration strategy dimension (collaborative-tasking versus individual-tasking)

Students sometimes worked in small groups or individually during this project. Initially, students seemed to spend similar amounts of time on collaborative versus individual tasks. Different students exhibited different patterns in using collaborative or individual tasking strategy. For instance, the data reported in the EmoForm indicates that student H4 did almost all of the tasks collaboratively, except for a short moment of individual tasking on the fourth day. On the contrary, student A1 spent most time single-tasking except for the first 2 days of collaborative tasking and a few moments of collaborative-tasking on the nineth day. Overall, we found that all students followed both of these strategies, and most of them worked increasingly in collaboration as the project progressed.

Students seemed to spend more time on collaborative tasks than individual tasks based on data collected from the first section of EmoForm. Table 6 shows how students relatively spent relatively more time on collaborative tasks (see activities with boldface numbers) such as empathizing design user (EDU), ideating design solution (IDS), making prototype (MP), and design documentation (DD). Besides, students often engaged with multi-tasking activities collaboratively. Likewise, for individual tasks, as shown in Table 7, students tend to spend more time on empathizing design users (EDU), ideating design solutions (IDS), and making prototype (MP). Comparing the time spent on individual versus collaborative tasks (Table 6 versus Table 7), it seems that some tasks favoured individual works, such as defining the design problem (DDP), presentation (PP), and planning (PL). In some rare instances, students were involved in interleaved collaborative and individual work within a single 25-min timeslot. For example, a student spent half of the time collaboratively getting feedback from others collaboratively and individually for the remaining time.

These findings above are related to the social environment of DBL, which are driven by the peer learning process within and across teams when they share resources, engaging in debate, and exercise freedom in task implementation (Gómez Puente et al. 2013a, b). In this paper, we address the collaboration strategy dimension of DBL activity distinguishing between three strategies; Collaborative task (CT), Individual task (IT), and intertwined IT and CT (intT) in this paper.

Specifically, we refine that the collaborative task refers to small groups of students working together to achieve the same task goal or working together to finish assigned tasks. In contrast, individual tasks are situations when students work on tasks alone. In intertwined individual and collaborative tasks, students interleave individual and collaborative tasks frequently. As shown in the example of Fig. 1, the student is intertwining individual and collaborative tasks regarding empathizing design user and ideating design solution between the minute 125 and minute 150.

A conceptual framework: the activity-and-affect model of DBL

The Activity-and-Affect Model of DBL (as shown in Fig. 4) was synthesized from the results in this study to capture how students experience emotions during DBL activities (to answer RQ1 and furtherly understand RQ2 synthetically). It is intended as a conceptual model, and it expands upon earlier descriptions of the DBL process such as the DBL framework (Gómez Puente et al. 2013a, b), the Reflective DBL framework (Bekker et al. 2015), and the Learning-by-Design framework (Kolodner et al. 2003). Additionally, the Activity-and-Affect Model of DBL is proposed to address the following two intentions that are underrepresented in the existing literature:

• (1) Describing the DBL activities from a multi-dimensional perspective. More specifically, all these vital elements of DBL are mapped along the task dimension, task strategy dimension, and collaboration strategy dimension, respectively.

• (2) Having a nuanced view of how a specific activity could be associated with an emotional experience. This model establishes the nuanced channels between DBL activities and students’ emotional experiences.

As shown in Fig. 4, this model introduces three dimensions of DBL activities using three identified colours. The blue rectangle block represents the task dimension, which includes three categorized tasks (see Table 4); the yellow circle and red circle represent the task strategy dimension and collaboration strategy dimension, respectively. These circles and rectangles presented with identified abbreviations and colors function as the foundations for constructing DBL activities from three dimensions. Further, every loop (connecting three nodes from each of the three dimensions, respectively) in this model represents a possible type of activity in DBL. For example, the loop on edge stringing with the nodes of “IT-MT-DTP” stands for the activity that individually involved multiple tasks in parallel to a design thinking process. In general, the arc connecting nodes represent one indicator of the emotional experience (e.g., enjoyment, frustration, etc.; see Tables 1 and 2 for the indicators of students’ emotional experience). For example, the feeling of enjoyment when mixing multiple (task strategy dimension: MT) design stages (task dimension: DTP) in teamwork (collaboration strategy dimension: CT) is a part of a student’s emotional experience of DBL. Likewise, the feeling of boredom when mixing multiple (task strategy dimension: MT) design stages (task dimension: DTP) alone (task strategy dimension: IT) is also a possible part of a student’s emotional experience of DBL.

The discussion above introduced the Activity-and-Affect model of DBL to describe students’ emotional experiences of DBL activities. Potentially this model is intended to explain students’ emotional experience in DBL. For this purpose, we fitted a linear regression model to the data collected by EmoForms from a sample of 30 students on a repeated basis adhering to the taxonomy of our proposed Activity-and-Affect model of DBL. Specifically, we performed a linear regression using a hierarchical data structure (i.e., a linear regression-based analysis that takes the hierarchical structure of the data into account) to explain individual student’s emotional experiences from DBL activities. Therefore, we sorted the data set collected by EmoForms into a three-level nested structure:

• (1) Level 1 (activity level) measurement occasions, i.e., DBL activities (where is coded as the dimension of task, task strategy, and collaboration strategy according to the structure of Activity-and-Affect model of DBL) nested within-day within-person.

• (2) Level 2 (day level) repeated measurement nested within-day (which is measured as fifteen different lessons/days) within-person.

• (3) Level 3 (student level) repeated measurements nested within-person (which is measured as 30 students).

The three-level multilevel multiple linear regression was calculated to explain six dependent variables of emotional experience (including enjoyment, relaxation, frustration, boredom, concentration, and learning better respectively) based DBL activities within days within-person. It is important to note that only one-time measurement rather than repeated measurement is designed within an occasion for this set of outcome-focused emotions (as seen in the last section of EmoForm in Fig. 1). Therefore, the two-level regression was calculated to explain four dependent variables of emotional experience (including contentment, pride, anxiety, and hopelessness). Overall, the results indicate that a regression model using a multilevel nested structure was a significant predictor of individual student’s emotional experience within a day (as seen in Table 8, all outcome variables of emotional experience having the value of R² > .5).

For example, the results (as seen in Table 8) indicate that a three-level structured regression model can significantly explain 84.8% of enjoyment variance, F (571, 1114) = 10.904. The level of enjoyment is significantly dependent on individual student (level 3), $\eta_{p}^{2}$ = .695, and student’s enjoyment is dependent on the day (level 2) on which the activity took place, $\eta_{p}^{2}$= .723. Specifically, the level of student’s enjoyment is dependent on both of the task (level 1) and collaboration strategy (level 1) of activity within a day, $\eta_{p}^{2}$ = .047, and .129, respectively.

This three-level structured regression model can significantly explain 87.6% of relaxation variance, F (566, 1099) = 13.714. The level of relaxation was found to be dependent on individual students (level 3), $\eta_{p}^{2}$ = .736, and student’s relaxation is dependent on the day (level 2) the activity took place, $\eta_{p}^{2}$ = .746. More specifically, student’s relaxation within a day was greatly influenced by the three elements of an activity: task (level 1), task strategy (level 1), and collaboration strategy (level 1), $\eta_{p}^{2}$ = .033, .045 and .085 respectively.

Likewise, the three-level structured regression model can significantly explain 85.2% of boredom variance, F (563, 1100) = 11.239. The level of boredom is significantly dependent on individual students (level 3), $\eta_{p}^{2}$ = .629, and student’s boredom is dependent on the day (level 2) the activity participated in, $\eta_{p}^{2}$ = .754. Moreover, all three aspects of the activity, including task (level 1), task strategy (level 1), and collaboration strategy (level 1), greatly influenced student’s boredom within a day, $\eta_{p}^{2}$ = .033, .044, and .109 respectively.

With regard to frustration, results indicate that the three-level structured regression model can significantly explain 84.6% of frustration variance, F (562, 1101) = 10.767. The level of frustration is significantly dependent on individual students (level 3), $\eta_{p}^{2}$ = .616, and student’s frustration is also dependent on which day the activity occurred, $\eta_{p}^{2}$ = .749. More specifically, the task (level 1) and collaboration strategy significantly account for student’s frustration within a day, $\eta_{p}^{2}$ = .044, and .082, respectively.

Our three-level structured regression model significantly explains 79.9% of concentration variance, F (572, 1117) = 7.750. The level of student’s self-perception of concentration was found to be significantly dependent on individual students (level 3), $\eta_{p}^{2}$ = .555, and student’s concentration is dependent on the day on which activity took place, $\eta_{p}^{2}$ = .672. Additionally, both the task (level 1) and collaboration strategy (level 1) of activity significantly account for student’s self-perception of concentration within a day, $\eta_{p}^{2}$ = .029, and .086, respectively.

Similarly, the three-level structured model significantly explains 83.6% of learning better variance, F (570, 1100) = 9.893. The level of perception of learning better is significantly dependent on individual students (level 3), $\eta_{p}^{2}$ = .595, and student’s perception of learning better is significantly dependent on the day that an activity took place, $\eta_{p}^{2}$ = .718. Specifically, student’s perception of learning better within a day was greatly influenced by the three elements of an activity: task (level 1), task strategy (level 1), and collaboration strategy (level 1), $\eta_{p}^{2}$ = .069, .063 and .093 respectively.

In terms of four outcome-related achievement emotions (e.g., contentment, pride, anxiety and hopelessness), the two-level structured regression model significantly explains 60.9% of contentment variance, F (161,248) = 2.403; 64.0% of pride variance (F (161,248) = 2.740); 52.9% of anxiety variance (F (161,248) = 1.728); and 61.2% of hopelessness variance (F (161,247) = 2.419) respectively. Furthermore, the level of contentment, pride, anxiety and hopelessness are significantly dependent on individual student (level 3), $\eta_{p}^{2}$ = .301, .342, .196 and .289 respectively.

Relationship between students’ emotional experience and DBL activities (RQ2)

In the previous section, our results suggest emotional experience (including a total of ten dependent variables as summarized in Tables 1 and 2) in DBL is significantly dependent on individual students (level 3). Student’s enjoyment, relaxation, frustration, boredom, concentration, and self-perception of learning better are dependent on the day when students took part in an activity (level 2). At the activity level (level 1), the multilevel regression results only indicate the general type of activity (from the dimensions of the task, task strategy, and collaboration strategy, respectively) in DBL.

To answer how specific DBL activities are related to the emotional experience (RQ2), we conducted multiple linear regression using a stepwise method to furtherly measure which DBL activities significantly contribute to students’ emotional experience. Specifically, this analysis investigates the fine-grained types of DBL activities from the dimensions of the task, task strategy, and collaboration strategy, respectively. For instance, the task dimension is coded as the different sub-tasks (as summarized in Table 4). The task strategy dimension is coded as single-tasking and multi-tasking. Besides, the collaboration strategy dimension is coded as individual tasking, collaborative tasking, and intertwined individual and collaborative tasking. All the multiple regression analyses were calculated on a group level, rather than distinguish individual differences. The detailed results regarding multiple linear regressions are displayed in “Appendix 3” (see Table 9).

Overall, the descriptive results of each variable showed that students had a positive experience in DBL according to the low scored negative emotions and high scored positive emotions. For instance, the mean scores for anxiety (M = 1.16, SD = .56; N.B. score “1” as “not at all” and “5” as “very much”), hopelessness (M = 1.47, SD = .92) and frustration (M = 1.61, SD = .99) are low. The positive indicators of emotional experience, e.g., enjoyment (M = 3.24, SD = 1.16), contentment (M = 3.56, SD = 1.13), pride (M = 3.18, SD = 1.28) and self-perception of concentration (M = 3.50, SD = 1.11) had a relatively high scores. The remaining three indicators of emotional experience are to different extents near to the middle point of the scale, including boredom (M = 2.47, SD = 1.31), relaxation (M = 2.82, SD = 1.21) and self-perception of learning better (M = 2.32, SD = 1.11).

Getting feedback from others (GST) has a positive effect on student’s relaxation (β = .101). Additionally, students who were busy with making prototype (MP) report a higher level of enjoyment (β = .129), relaxation (β = .086), pride (β = .133) and self-perception of concentration (β = .201) but also a lower level of boredom (β = − .198) and hopelessness (β = − .130). A mixed design thinking steps and project management (%DTP-PM) is a specific type of task where applying a multi-tasking strategy was found to have a positive effect on student’s self-perception of learning better (β = .050) but a negative impact on student’s boredom (β = − .054). Likewise, students using the collaborative-tasking strategy (CT) report a higher level of enjoyment (β = .112).

Some other tasks during the design thinking process seem to have a negative relationship with students’ emotional experience. For example, students who were involved in the task of empathizing design user (EDU) report a lower level of relaxation (β = − .062), contentment (β = − .197), and in the meanwhile report a higher level of frustration (β = .157) and boredom (β = .148). Students working on the task of defining design problems (DDP) indicate a lower level of student’s enjoyment (β = − .122), and a higher level of boredom (β = .075) and hopelessness (β = .109). Students, when busy with testing a prototype (TP), report a lower level of contentment (β = − .132) and a higher level of frustration (β = .070). Similarly, presentation (PP) and design documentation (DD) are the two project management tasks that both seem to be positively related to frustration (β = .082; β = .049, respectively). Besides, when working on design documentation (DD), students report a lower level of relaxation (β = − .064). In comparison, when busy with presentation (PP), students indicate a lower level of enjoyment (β = − .049) and a higher level of anxiety (β = .148).

The combined design thinking process and task-related social interaction (%DTP-TSI) is another specific type of task in which particularly applying a multi-tasking strategy was found to make students feel a higher level of anxiety (β = .170). Moreover, the single-tasking strategy (ST) seems to have a negative effect on student’s enjoyment (β = − .138), pride (β = − .339), self-perception of concentration (β = − .163), and self-perception of learning better (β = − .158). Students using the individual-tasking strategy (IT) report a lower level of self-perception of concentration (β = − .074).

Interestingly, students involved in the task of planning (PL) report a lower level of boredom (β = − .061) and a lower level of self-perception of learning better (β = − .058). The combined design thinking steps (%DTP) seemed to be negatively related to both boredom (β = − .071) and pride (β = − .127). Similarly, students, when getting support from the teacher (GST) and involving a particular task of mixed design thinking steps and task-related social interaction (%DTP-TSI), report a higher level of relaxation (β = .075, β = .090 respectively) and also higher level of frustration (β = .049, β = .231 respectively). Furthermore, getting support from the teacher (GST) seemed also to be positively related to boredom (β = .059).

Discussion and conclusion

In summary, our results suggest future work should seek a more fine-grained understanding of the dynamic DBL process since some potential issues need to be clearly articulated, such as the level of iteration and multi-tasking. Besides, our results also suggest that students and educators should have a flexible approach towards the student’s emotional experience. It is because there is a subtle relationship between students’ emotional experience and different DBL activities that emotion may change along with moving among tasks or strategies.

A more nuanced understanding of DBL context, more insights

This paper presents empirical evidence of the variety of ways in which students go through a sequence of DBL tasks. For example, as shown in Fig. 2, students made several transitions between empathizing (EDU) and ideating (IDS); however, not every student spent time reframing design problems (DDP). Few students interleaved making (MP) and testing (TP), while a few students seldom interleaved ideating (IDS) and building (MP). They mostly followed up with making (MP) and testing (TP) for the later parts of the project and uniformly ended the project by preparing and presenting (PP) their work, which consistently with the teacher’s plan for the class and to enable assessment. Getting support from the teacher (GST) and getting feedback from others (GFO) occur at different times during the project with no apparent pattern, most often in parallel with other activities. This can be because different teams and individuals do not follow a strict synchronized process, so interactions happen serendipitously. Also, individuals may need help at very different moments depending on their abilities, their task, and their learning. Students in our case spent more time on design thinking steps such as empathizing (EDU), ideating (IDS), and prototyping (MP).

Even though they went through the design process in a relatively consistent way, some individuals do more iterations than others over time. It has been argued before (Hmelo et al. 2000) that the most crucial element for successful learning in DBL is to have multiple iterations towards a solution, as iteratively generating solutions help understand the aspects of the problem that need to be considered (Kolodner and Wills 1996). However, no clear criterion has been provided for the early iterations in DBL (Adams and Atman 2000), partly due to the ambiguous nature of ill-defined design problems and partially because the reframed problem and its solution are developed concurrently (Wynn and Eckert 2017). This raises questions about how DBL educators can encourage iteration to ensure effective learning. One may argue that the process of frequently switching tasks when applying a multi-tasking strategy in DBL would potentially motivate students’ iterative inquiry. This would suggest further investigation of the correlation between multi-tasking and iteration in DBL.

Despite that multi-tasking is not a new phenomenon as such, there has not been an earlier attempt to describe how students multi-task during DBL. Task switching has been recognized as a critical element of multitasking (Spink et al. 2008). The present study explicitly mapped students’ frequent task-switching in DBL and particularly discerned six ways in which students worked on at most two to three tasks within 25 mins interval period. The two most frequent of which among these six ways were: (a) combining different design process steps, and (b) mixing design process steps and task-related social interaction. Zooming in on how these tasks were related, we note that empathizing design user (EDU), ideating design solution (IDS), getting support from the teacher (GST), and getting feedback from others (GFO) were often combined. We would suggest researchers interested in multi-tasking in DBL carefully examine the trade-offs involved in multi-tasking. Future research could pay closer attention to whether and how different multi-tasking activities (e.g., the co-occurrence of EDU and IDS, or IDS and GFO) influence the learning process. Besides, future research could investigate the correlation between the level of experience in DBL and the strategy of multi-tasking.

Our findings suggest that specific tasks may invite individual work, e.g., empathizing (EDU), ideating (IDS), and prototyping (MP). However, other tasks may favor collaboration, e.g., defining (DDP), presentation (PP), and planning (PL). Students sometimes work individually or in small teams, and this case allowed us to observe how different individuals use different collaborative or individual tasking strategies. A general pattern across individual participants was that as the project progressed, students worked increasingly in collaboration. Future research could attempt to derive guidelines for DBL teachers regarding how to steer students in allocating their effort in individual or joint tasks.

The variable impact of emotions

Overall, DBL activities across the project were rated positively. Especially, prototyping is an appealing and enjoyable task in DBL. We found a positive relationship between prototyping and positive emotions, e.g., enjoyment and pride, which also is consistent with earlier findings (Zhang et al. 2018). Besides, our results indicate that prototyping is positively related to students’ self-perceived concentration levels. Reflecting on the positive impact of prototyping on students’ emotional experience, this would suggest DBL educators could orchestrate the session of prototyping as a moment for students to engage and enjoy. On the other hand, getting feedback from others, as a way of receiving summative assessment in DBL (Gómez Puente et al. 2015), was found to be positively related to relaxation feelings, which can also facilitate learning evaluation. Future design considerations in terms of DBL classroom management may foster students’ relaxation and reflection by facilitating feedback moments in groups.

Individual work seemed to be negatively related to students’ self-perceived concentration levels. On the contrary, teamwork was consistently rated positively, in line with other earlier research, e.g., (Carroll et al. 2010). However, the collaborative aspect of teamwork is still challenging for some students in DBL, as argued in one study (Neve and Keith-Marsoun 2017). This would suggest that DBL teachers should facilitate the classroom culture of teamwork (Doppelt 2003) and also encourage the transition to teamwork when individual tasks appear to be experienced negatively.

Mixing design thinking steps with project management is a type of task as a multi-tasking strategy, which seemed to increase students’ self-perception of learning better and decreased boredom. This justifies investing some effort in tracking progress to be able to take charge of the project and also to move back and forward in the design process. Therefore, future design research in DBL could focus on developing tools for scaffolding students’ planning and task management.

Earlier research provides no clear answer as to whether learning is more effective while students are in a positive or in a negative emotional state, as the answer largely depends on the nature of the learning task (Fiedler and Beier 2014). Specifically, confusion or frustration may be beneficial for learning (D’Mello et al. 2014), especially in DBL, (as opposed to traditional learning environments involving passive listening lectures, take examinations and individual assignment, etc.). One notes that the frustration by the failure of testing and building design ideas may motivate students to find new solutions (Zhang et al. 2020), which argues for flexibility in dealing with the negative emotions of students in DBL. Moreover, the same task could have utterly different effects depending on the collaboration strategy adopted.

One limitation of this study concerns the potential inconsistencies in students’ recording of data. Although we’ve used mixed methods of data collection (self-reporting surveys, classroom observations, and interviews) to triangulate the results, the risk of inconsistencies in data recording still may exist. It is due to the observation measurement in this study was conducted at a group level while the self-reporting survey was at an individual level. The strategy we’ve applied to fix the inconsistency is to focus more on the self-reported data while using observations to get more insights on the contextual information and using the interviews to verify the data collected in field observations and self-reporting surveys. We would suggest future work to resolve the potential inconsistency issue by testing the trustworthiness of data collected, which is equally important as using multimodal data collection to triangulate results. Another limitation concerns that the participants in this study already had experience with DBL for 1 year. This may have affected the results. Specifically, we could expect that for students experiencing DBL for the first time, there could be a novelty effect that could potentially result in more positive experiences and a different execution of the design process. In contrast, one strength of the present study is that such a novelty effect does not confound the results. Future research could examine how activities affect participants’ emotions for groups who are at varying levels of experiencing DBL (e.g., who are novices or experts to DBL).

Appendices

Appendix 1: Project information

Design challenge

Fire can cause a lot of damage and suffering. You can prevent fire or limit the damage with a few simple measures. Unfortunately, many people do not know what actions they should take if there is a fire. For tips on fire safety in and around the house, (See this link https://www.brandweer.nl/brandveiligheid).

Escape rooms have become enormously popular. The Eindhoven fire service wants to introduce people to fire safety in and around the house in a modern and interactive way via an escape room. Together with other people, you will be stocked in a place and try to escape from the room through instructions and solving puzzles.

Design client

You have been hired as a game designer to design an escape room for the fire brigade of Eindhoven. The client is Mr. Peter van der Horst from the Eindhoven fire department.

Assignment

Design a mobile escape room that allows participants to get acquainted with fire safety in and around the house. This mobile escape room must fit into a sea container so that it can be deployed at different locations.

There are several conditions:

• A group of participants must be able to escape in half an hour.

• At least two puzzle assignments must be mechanical and/or electronic.

Assignment guide

To achieve a good result, you may need to go through several steps:

• Step 1 Brainstorming. What do you already know about the subject? Which ideas come to you immediately? The client asks for a mind map of the ideas that the assignment evokes.

• Step 2 Professional training. You will work as a game developer during this assignment. The client wants you to briefly describe what exercise you can do to become a game developer and what the training entails. Also, he wants you to explain what this profession involves and what the activities are.

• Step 3 Understanding technical knowledge. The client wants to be informed about the professional language of the game developer. He expects to know the technical terms in this field and an explanation of how you apply them in the assignment.

• Step 4 Researching escape rooms. The client wonders which game elements provide an ultimate escape room experience. To answer this question, the client expects you to analyze several existing (educational) escape rooms. The client wants to see an analysis of the storyline, the plot, the operation of the ‘puzzles,’ and the atmosphere worked out.

• Step 5 Researching fire safety. What knowledge about fire safety do you want to impart to the visitors of the escape room? The client is curious about what themes around fire safety you find suitable as a red thread for the escape room (for example, setting up a safe house or what you can do when there is danger).

• Step 6 Identifying design requirements and parameters. The client wants a set of requirements and design parameters of the escape room based on your preliminary investigation. He expects to see an overview in which you indicate what parameters and conditions are set for the form, the content, and the use of the escape room.

• Step 7 Ideating a storyline. A strong story with a good plot is a basis for an escape room. This story connects the puzzles and ensures that the game runs logically. The client challenges you to come up with an engaging storyline that informs visitors about your theme within half an hour. What surprising twists are there? What links are there in the story that visitors have to discover? It is expected that step by step is explained what happens in the story.

• Step 8 Sketching design puzzles. “There are many ways that lead to Rome.” Your story can still be worked out in various ways. For every step in the story, several puzzles can be considered. The client wants to see a brainstorm of puzzles per step in the story of every team member. You can refer to the ideas and puzzle forms from Escape rooms that you have found as an example in the preliminary investigation. Sketches and captions will make clear how the puzzle works, and which materials are used.

• Step 9 Designing. There are now several (three to four) ideas per step in the game! It is up to you to respond to each other’s ideas in a constructive manner and to choose the best puzzle per step in the story or to make a combination of good ideas. The client expects that the red thread (storyline) comes to the fore through the choice of the puzzles, a good balance of the type of games, and a lot of variety (physical, thinking, mechanical, electronic, etc.). To be able to assess everything accurately, the client wants to see an overall picture of the course of the game and the puzzles that will be discussed. He expects you to explain in the file as visually as possible how the game progresses. He also likes to see an explanation of why your game meets the design requirements.

• Step 10 Prototyping. The client wants to work out at least two puzzles in the form of a working prototype (preferably scale 1: 1). At least one prototype is mechanical and/or electronic. The client expects technical drawings (dimensions in millimeter) and a working drawing so that he has a good view of the requirements for making the puzzle.

• Step 11 Completion of the assignment. The project is completed with a video presentation of the escape room in action. With this, the client has to get a good picture of your escape room. Show how each puzzle works and how it fits into the storyline. How does your escape room ensure that people gain relevant knowledge and found it fun to do?”

Appendix 2: Example of prototypes

Students’ puzzle design prototypes addressing the escape room design challenge in this DBL project are varied. For example, one group used the Littlebits to make a system that unlocked a door by a sensor. Some created traditional puzzles (e.g., the case of testing such a prototype in the photo below), while some invented informative puzzles. Some groups used accessible electronics (e.g., lights) to design their interactive puzzles. Some used physical principals (e.g., magnetism) to get a key out of a tube by a magnet.

Example of testing a puzzle prototype.

Appendix 3

See Table 9.