1 Introduction

The COVID-19 pandemic led the field of teaching and learning to experience a rapid and significant transformation due to the adoption of distance learning (Mesghina et al., 2021; Wong et al., 2023a, b). As the pandemic forced social distancing measures nationwide (CDC, 2020; The White House, 2020), various sectors, including education, had to quickly adapt their workflows. Educational establishments encountered the task of completely transforming their course delivery framework within a few weeks while temporarily halting in-person instruction (Ferrel & Ryan, 2020). As a reaction to the pandemic, numerous institutions turned to synchronous online video-conferencing sessions using platforms such as Zoom as an short-term emergency measure to maintain the ongoing process of teaching and learning. However, it is essential to recognize that this approach was intended for short-term use and should not be equated with online learning experiences based on evidence-based pedagogical learning design. The onset of the COVID-19 crisis mandated a deliberate shift in the methods of delivering courses, underscoring the necessity for more research to understand key characteristics of effective online learning experiences.

Zoom undoubtedly facilitated teaching and learning, but it also brought about concerns regarding student disengagement, lack of interest, and validity from both instructors' and students' perspectives (Son et al., 2020; Unger & Meiran, 2020; Wong et al., 2023a, b). Concurrently, students reported various challenges related to technical difficulties, difficulty concentrating, increased distractions, and decreased motivation during emergency remote distance learning through synchronous online video-based instruction. These issues significantly impacted the student learning experience (Agarwal & Kaushik, 2020; Fawaz & Samaha, 2021; Kaharuddin, 2020; Son et al., 2020). Thus, the focus of this study is to gain insight into how designers may address these challenges by shifting from synchronous video-conferencing courses to asynchronous video-based online math courses. This study investigates the opportunities for student engagement within an online math course grounded in the learning experience design (LXD) framework. The aim is to investigate the frequency of mind-wandering and online engagement among children in this context.

In this research, the implementation of Learning Experience Design (LXD) involved the integration of e-learning instructional design frameworks and the meticulous crafting of user experience design for the online course. The investigation delves into the intricate relationship between students' situational interests and mind-wandering within the online learning setting, with a focus on how LX designs can either facilitate or impede students' engagement in the online environment. This study contributes to the comprehension of evidence-based asynchronous online learning contexts firmly rooted in LXD, demonstrating its viability and efficacy as an approach for children learning mathematics. The primary objective is to glean insights into the ways in which learning experience design initiatives can elevate the engagement levels of young students participating in an online math course aligned with the principles of LXD.

2 Theoretical framework

The subsequent sections summarize key aspects of the existing body of LXD and cognitive literature that were integrated to develop, and test the theoretical foundation for our model. Additionally, their practical implications in the realm of teaching and learning are emphasized given the scarcity of research that has delved into the interconnectedness of these constructs, prompting thorough analysis. This comprehensive review of the literature serves as the foundation for our proposed model, aiming to delineate the experiences of young students engaged in online learning.

2.1 Learning experience design

Learning Experience Design (LXD) involves crafting learning scenarios that extend beyond the confines of the traditional classroom environment, often harnessing the capabilities of online and virtual technologies (Ahn, 2019). It encompasses a process of crafting impactful learning encounters that prioritize human-centric design and goal-driven approaches, all aimed at attaining specific educational objectives (Floor, 2018; Wong & Hughes, 2023). Diverging from the conventional concept of "instructional design," which predominantly revolves around curriculum development and instructional strategies for knowledge acquisition (Correia, 2021), LXD, as articulated by Weigel (2015), represents an interdisciplinary amalgamation that integrates instructional design, teaching pedagogy, cognitive science, learning sciences, and user experience design. Practically, LXD involves five key aspects:

  1. 1.

    Human-Centered Approach: Learners' needs, preferences, and perspectives are prioritized to create tailored experiences (Matthews et al., 2017; Wong & Hughes, 2023).

  2. 2.

    Goal-Oriented Design: Learning experiences are carefully designed to achieve specific objectives, making them purposeful and relevant (Floor, 2018; Wong & Hughes, 2023).

  3. 3.

    Grounding in Learning Theory: Evidence-based practices from cognitive science and learning sciences inform the design process for effectiveness (Ahn, 2019).

  4. 4.

    Emphasis on Learning Through Practice: Active and experiential learning methods, such as hands-on activities and real-world applications, deepen understanding and retention (Floor, 2018; Wong & Hughes, 2023).

  5. 5.

    Interdisciplinary Collaboration: LXD brings together professionals from diverse backgrounds, including instructional designers, educators, cognitive scientists, and user experience designers, to create holistic and well-rounded learning experiences (Weigel, 2015).

Each of these facets emphasizes empathy, wherein the intended and unintended learning design outcomes are carefully considered to support learners' experiences (Matthews et al., 2017; Wong & Hughes, 2023). Consequently, LXD expands the definition of learning experiences, enabling instructors and designers to empathize with learners and enhance the learning design toolbox (Ahn et al., 2019; Weigel, 2015). As educational practices evolve, LXD empowers educators to adapt and improve their methodologies, leading to successful and enriched learning outcomes (Ahn, 2019; Floor, 2018; Wong et al., 2021).

2.2 Engagement

Within the realm of education, student engagement pertains to the degree of exertion and active participation exhibited by students during their learning endeavors (Hu & Kuh, 2002; Lo & Hew, 2020; Richardson & Newby, 2006). In the domain of online courses, engagement transcends mere effort, encompassing facets like attention, inquisitiveness, interaction, and the interests that students demonstrate throughout a learning module. Moreover, it encompasses the motivational aspects that students may showcase during the learning journey (Luan et al., 2023; Pellas, 2014). Research in the field of online learning has linked increased student engagement to several factors. One key factor is the quality of instructional design (Marrongelle et al., 2013; Pappas, 2015). Furthermore, student engagement can be impacted by their interaction with the course user interface (Hu, 2008), along with diverse motivational elements like interest and self-efficacy, often originating within the learning environment (Chen et al., 2001; Sun & Rueda, 2012). When learning interfaces prove challenging to navigate, lack engagement, or fail to spark students' interest, studies have demonstrated that this can lead to negative learning experiences. These experiences may encompass increased occurrences of mind-wandering, where students shift their attention away from the primary learning task (Desideri et al., 2019). Therefore, to foster students' interests and enhance engagement, our focus lies in ensuring quality instructional design and providing a strong user experience that maximizes students' participation and degree of interactivity. By prioritizing these design efforts, we aim to create an optimal learning environment that promotes active involvement, curiosity, and motivation among students in our online courses.

2.3 Mind-wandering

Mind-wandering, colloquially referred to as "zoning out," describes the occurrence when an individual's focus shifts from the main task to internal thoughts (Singer, 1975; Smallwood & Schooler, 2006; Wong et al., 2023a, b). It is the experience of mentally wandering away from the present moment, away from the "here and now" (Smallwood & Schooler, 2006). In contrast to a state of focused and concentrated attention, mind-wandering is described as an off-task cognitive state marked by instances of attentional lapses (Carriere et al., 2013; Danckert & Merrifield, 2018). This inclination is more prone to manifest in situations of monotony (Eastwood et al., 2012) or extended periods of cognitively undemanding tasks (Smallwood & Schooler, 2015). During the COVID-19 pandemic, Conrad and Newman (2021) undertook research which unveiled that students who frequently experienced mind-wandering during extended Zoom class sessions tended to exhibit lower achievement scores. As Zoom classes often involve low cognitive demand and may be associated with boredom, frustration, and disengagement (Hodges et al., 2020; Katz & Kedem-Yemini, 2021; Wong et al., 2023a, b), it is crucial to identify factors that contribute to mind-wandering (Randall, 2015; Schooler et al., 2011; Smallwood et al., 2007).

Mind-wandering is interpreted as an indication of a pause in the processing of information, involving a transition of focus from external task-related content to internal contemplation of thoughts unrelated to the ongoing task (Smallwood & Schooler, 2015). As an illustration, a student might be engrossed in a video and then unexpectedly find their attention diverted by elements in the surroundings, conflicting attractions, or a lack of engagement with the course content, among other possibilities. This shift of attention is regarded as a disengagement process between the external task and an individual's pre-existing mental framework (self-generated thoughts) (Mills et al., 2013; Smallwood et al., 2007). According to the Current Concerns Hypothesis, mind-wandering arises when the reward for contemplating personal goals surpasses the reward for engaging with the primary objectives of the task (Klinger, 2009). A study by Wong et al. (2023a, b) found that learners engaged in online synchronous Zoom learning who held strong task-value beliefs and self-efficacy demonstrated lower levels of mind-wandering and higher levels of online engagement. To reduce mind-wandering and enhance students' motivation to learn, Wong et al. (2023a, b) proposed specific strategies, including intentional learning design, pedagogical frameworks, and multimedia learning principles to improve the online learning experience. Consequently, when assessing students' values in a teaching and learning context, considering their situational interests during an online learning experience becomes essential.

2.4 Situational interest

Situational interest (SI) encompasses the curiosity and fascination that stem from the specific learning environment or the unique nature of a particular learning task (Chen et al., 2001; Schraw & Lehman, 2001). Schraw et al., (2001) argued that fostering students' SI during learning enhances their intrinsic motivations and encourages the use of various learning strategies. This indicates that SI is flexible and can be increased based on the learning environment and the specific task at hand. Past research has shown that situational interest as a potent motivator in math, reading, and history, particularly when learners actively participate and engage throughout the learning process (Chen et al., 2001). While the relationship between situational interest and student engagement has been extensively explored in traditional in-person learning settings (Sun & Rueda, 2012), there is less understanding of how to cultivate situational interest for young students in online learning environments while also reducing mind-wandering tendencies. In response to this gap, we hypothesized that implementing the Learning Experience Design (LXD) paradigm might offer valuable support to elementary students in adopting an asynchronous online learning approach. By leveraging LXD principles, we aim to create an engaging and effective online learning environment that sparks students' situational interests, thereby enhancing their motivation and reducing the occurrence of mind-wandering. This exploration of the LXD paradigm presents an opportunity to optimize the learning experiences of young students in the context of asynchronous online learning.

2.5 User experience

User experience (UX) can play a vital role in shaping learners' situational interest and engagement within a specific learning interface. User experience is defined as the extent to which a product allows specified users to achieve their goals effectively, efficiently, and with satisfaction within a defined context of use (ISO 9241-11, 1998). Within the framework of an online course, UX pertains to the efficacy of the learning interface and its capacity to facilitate students' seamless interaction with the course platform, allowing them to effectively achieve their desired objectives. Learning environments with strong UX are designed in a way that directs students' focus toward the learning task itself, rather than struggling to navigate the content (Wong & Hughes, 2023). This shift in perspective compels researchers and instructors to not only focus on the instructional content but also deeply understand users— their needs, preferences, abilities, and the limitations of the learning interface. Hence, the goal of UX in an online course is to craft a positive and enriching learning journey for students. This involves ensuring content accessibility, nurturing interest, and implementing effective interface design best practices, all of which have been identified as significant predictors of student engagement in online learning environments (Lynch & Dembo, 2004; Simunich et al., 2015). By prioritizing UX in online course design, educators can foster a more engaging and rewarding learning experience that ultimately benefits students' motivation and achievement.

2.6 Situated cognition theory

The instructional design framework chosen to support students' interests and create effective e-learning experiences during their transition to distance learning is the Situated Cognition Theory (SCT) (Ghefaili, 2003). The foundation of this framework centers on the concept that optimal learning is achieved through integration with real-world activities. By embracing SCT, our aim was to enable learners to comprehend concepts and skills within the very contexts they will be put to use (Brown et al., 1989). In practice, SCT underscores immersive learning environments, where new information is presented to learners in a manner mirroring real-life situations. To actualize this approach, we employed pre-recorded videos featuring authentic math lessons taught by an instructor in a genuine classroom setting. We integrated interactive components, including modeling, coaching, scaffolding, articulation, reflection, and exploration, with the intent of providing learners with significant and enriching learning encounters (Drijvers, 2019; Pappas, 2015; O’Brien & Battista, 2020). Through this SCT-based LXD, students engage with bite-sized video segments capturing authentic classroom interactions between a teacher and students. They then immediately apply what they have learned by working on scaffolded problem scenarios and providing written reflections on their approach to solving each math problem. The objective of this approach is to establish an immersive learning environment that promotes classroom ecological validity. This, in turn, empowers students to proactively cultivate their interests, diminish instances of mind-wandering, and actively engage within the confines of the online learning platform. By embracing the SCT within our LXD framework, our intention is to construct a learning experience that seamlessly integrates with the authentic experiences of students, cultivating a sense of pertinence and active engagement throughout their academic journey. This pedagogical approach empowers students to pragmatically apply their acquired knowledge within real-world contexts, thereby enriching their depth of understanding and motivation as they navigate the distinctive challenges inherent to the realm of distance learning.

3 Current study

Over the last couple of years, young students have embraced various "edtech tools" to facilitate their learning experiences, sometimes adapting to entirely new modalities of education. However, this shift in learning modalities has presented unforeseen challenges such as disengagement, mind-wandering, and interest (Agarwal & Kaushik, 2020; Son et al., 2020; Wong et al., 2023a, b). In this study, we leveraged evidence-based pedagogical LXD principles to support students' situational interest (SI) and incorporated human-centered user experience (UX) design to address technical concerns associated with distance learning. Our objective was to create a learning environment that fosters interest and engagement while alleviating potential barriers related to the online format. Furthermore, we explored the relationship between situational interest and user experience in predicting students' engagement, while also considering the potential of our design decisions to reduce mind-wandering in an online course grounded in the LXD paradigm. By focusing on these elements, we sought to create an effective and engaging online learning experience for young students, enhancing their motivation, participation, and overall learning outcomes. This study is guided by the following research questions:

  • (RQ 1) To what extent do students’ user experience, situational interest, and mind-wandering directly affect students' online math engagement?

  • (RQ 2) To what extent do students' mind-wandering and situational interest mediate the relationship between students’ user experience and students’ perceived online math engagement?

  • (RQ 3) To what extent do situational interest and mind-wandering sequentially mediate the path from students’ user experience to online math engagement?

4 Methodology

4.1 Research design

This research study involved a comprehensive survey analysis that focused on elementary and middle school students. These students had the opportunity to engage in a CCSS-aligned math intervention that emphasized higher-order thinking (HOT) skills. This intervention took place during the challenging context of distance learning, which was necessitated by the COVID-19 pandemic. For this study, we collaborated with educators from 5 and 6th grade classrooms within two school districts situated in the Orange County region in California. Prior to the COVID-19 pandemic, the university had established a research-practice partnership (RPP) with these two districts, specifically working with teachers from two elementary schools. Through this RPP, we worked alongside teachers to develop and research an online math course that was tailored to match the needs of teachers and students during this sensitive COVID learning context. This partnership not only ensured a robust academic collaboration but also facilitated a seamless integration of the higher-order thinking (HOT) math intervention into the existing educational framework (See Mesghina et al., 2024; Wong et al., 2021; Wong et al., 2022). The careful selection of these districts and schools was driven by the existing relationship and shared goals of enhancing math education, making it a well-informed decision rather than a random selection. The schools and teachers were chosen based on their willingness to participate and their alignment with the study's objectives. This deliberate approach to selecting participants aimed to create a research environment that closely reflected the real-world educational challenges faced during the pandemic, allowing for a more nuanced analysis of the intervention's effectiveness with improved ecological validity.

This study received funding from both the National Science Foundation (NSF) and the Institute of Education Sciences (IES). In adherence to university and grant funder requirements, the university institution secured approval from its Institutional Review Board (IRB) to conduct human subjects research. The research was categorized as IRB-exempt, as the data collection process was conducted anonymously online, involving minimal risk to the participants. All participant data were recorded with the utmost confidentiality and anonymity, and the nature of the questions, topics, and content posed no potential harm to the students. This research protocol received formal approval from the university's ethics committee.

4.2 Participants

The participation encompassed three 5th-grade teachers, six 6th-grade teachers, and a teacher instructing a combined 5th/6th-grade class. This collaboration yielded a total of 12 classrooms, each accommodating an average of 26 to 33 students. The cohort of participating students, comprising a total of n = 303, presented a diverse composition. Among the participants, 56.1% identified as female, while 42.6% identified as male (as delineated in Table 1). The age range of the students spanned from 10 to 12 years old, with a notable majority clustered around the ages of 10 (42.6%) and 11 (55.7%). Moreover, the sample showcased a broad spectrum of racial and ethnic backgrounds. The composition was as follows: 22.6% identified as White, 3.8% as Black, 1.1% as Native American, 17.1% as Asian, 1.1% as Native Hawaiian, 21.0% as Latinx/ Latino, 17.9% as other, and 4.8% as Mixed.

Table 1 Sociodemographic characteristics of participants
Full size table

4.3 Data collection and instrumentation

Data for this study were collected during the Winter term of 2021, utilizing an online survey administered through the Qualtrics XM platform. This survey was promptly conducted immediately after the students' engagement in the online higher-order thinking (HOT) math intervention, which spanned over a two-day period. This meant that students filled out the surveys immediately after completing their last lesson. Having students answer the survey right after the lesson was an intentional research design decision in order to maintain the rigor, robustness, and quality of responses from students. When responding to all of the surveys, students were asked to reflect on their learning experiences about the online math intervention. Four questionnaires were administered to the participants which included the Situational Interest (SI) Scale, Standardized User Experience Percentile Rank Questionnaire (SUPR-Q), Mind-wandering Questionnaire (MWQ), and the Perceived Engagement Scale. To ensure reliability of the scales, we did not alter the question count or sentence structure of the questions. However, slight modifications were necessary such as replacing “classroom” with “online course” in order to reflect the context of the online math intervention of this study. To help students’ assess their judgements with each of the survey questionnaires, smiley face emojis were added to each of the anchors in order to help support young student’s judgments. This practice has been shown to be effective, especially in children and young adolescents as an accompanying rating scale for quantitative survey evaluations (Hall et al., 2016; Savage, 2018). Further details pertaining to descriptive analysis and scale reliabilities are available in Table 2.

Table 2 Descriptive statistics and correlations for study variables (N = 303)
Full size table

Given our research questions and objectives, which aim to understand the impact of implementing the pedagogical paradigm LXD on the online math learning experiences of students, we conducted an extensive review of the literature to identify suitable survey instruments. In our search, we set specific criteria. Firstly, we extensively explored the existing literature on math interventions that examined the aspects of mind-wandering, situational interest, user experience, and engagement. Notably, there is a scarcity of studies that have collectively examined these specific variables within the context of elementary online math education. However, our study builds upon the work of Wong et al. (2023a, b), in which they assessed a theoretical model for learning experiences using a structural equation model (SEM) to investigate how mind-wandering mediates students' engagement in online courses. While their study did not explicitly delve into the implementation of LXD as a pedagogical paradigm, the authors did suggest in their practical implications that LXD is a contemporary evidence-based instructional framework that educators should consider when designing future online courses in the post-COVID learning contexts. Therefore, we adopted the same measures of engagement and mind-wandering from Wong et al. (2023a, b) in our current study to explore this model further. Regarding situational interest, Lyons et al. (2018) conducted an assessment of elementary math situational interests through an in-person video-based math intervention. This in-person video-based math intervention served as the iterative precursor to our online course (Begolli & Richland, 2018), which was deployed to students in our present study. Consequently, we utilized a similar measure of situational interest in our current research. Lastly, we selected the SUPR-Q as our instrument for evaluating usability and user experience within the course. In contrast to the System Usability Scale (SUS), the SUPR-Q is specifically designed to offer a more comprehensive evaluation of the user experience, encompassing dimensions such as trust, appearance, and loyalty, in addition to usability (Lewis & Sauro, 2021; Sauro, 2015). SUPR-Q's capacity to provide more granular data allows for the independent analysis of various aspects of the user experience which proves invaluable for pinpointing and addressing specific areas for improvement. Therefore, our selection of instruments for this study is rooted in the works of previous researchers, the extension of theoretical models, and the influence of LXD on students' learning experiences.

This set of instruments included the Situational Interest Scale (Chen et al., 2001), which assessed students' levels of interest specifically in the online mathematics lesson provided. All items were rated on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Example items include: "What I learned in the math lesson is fascinating to me" and "What I learned in the math lesson can be applied to real life." Chen et al. (2001) reported an internal consistency coefficient of 0.90 while Lyons et al. (2018) reported 0.78. In this study, the Cronbach's alpha is 0.92. Additionally, the Standardized User Experience Percentile Rank Questionnaire (SUPR-Q) (Sauro, 2015) was employed to explore students' interactions with the online platform. The SUPR-Q instrument is a set of 8 standardized questions that is traditional used in the User Experience (UX) and Human-Computer-Interaction (HCI) fields to measure users' perceptions around four components of a website or interface including, usability, trust and credibility, appearance, and loyalty. All items are rated on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Sample items include “This online activity is easy to use (user-friendly)” and “I can trust the information on this website.” Sauro (2015) reported an internal consistency reliability of 0.86. Other researchers who have applied this instrument in the context of online education such as Wong et al. (2023a) and Lewis and Sauro (2021) report the internal consistency coefficient to be 0.87 and 0.88. In this study, Cronbach's alpha of SUPR-Q was 0.90 (See Table 2).

The assessment of students' mind-wandering during the online math lesson was conducted using the Mind-Wandering Questionnaire (MWQ), which was created by Mrazek et al. (2013). Mrazek et al. (2013) developed the MWQ in order to have broad applicability, testing the questionnaire on college, high school, middle school, and young adolescent students in the classroom. This questionnaire comprised of five items, and respondents provided their answers on a 6-point Likert scale, ranging from 1 (almost never) to 6 (almost always). Example statements in the questionnaire include "I tend to engage in mind-wandering during lectures or presentations'' and "I often find myself simultaneously listening with one ear and thinking about something else." Mrazek et al. (2013) report the internal consistency coefficient to be 0.85. Wong et al. (2023a, b) utilized the MWQ in an online setting and revalidated this measure reporting an internal consistency coefficient of 0.87. In this study, the Cronbach’s alpha of MWQ was 0.93 (See Table 2).

To evaluate students' perceived online engagement, we employed the 12-item Perceived Engagement Scale Rossing et al. (2012). Response choices were provided on a 5-point scale, ranging from 1 (strongly disagree) to 5 (strongly agree). This instrument encompassed inquiries related to students' perceptions of their learning experiences and their sense of engagement in an online course. Sample items in the scale included statements such as "This online activity motivated me to learn math more than being in the classroom" and "Online video lessons are important for me when learning math at home." Rossing et al. (2012) report that the internal consistency coefficient for this instrument was 0.90. Similarly, Wong et al. (2023a, b) reported a coefficient of 0.88, further supporting the scale's reliability across online learning contexts. In this study, the Cronbach’s alpha of the perceived engagement scale was 0.92 (See Table 2).

4.4 Video design of math lesson

We first worked with teachers to align with their existing Common Core State Standards (CCSS) math lesson plans. After understanding how we could best support teachers in the classroom, we then proceeded to co-develop the video lesson. Teachers, researchers, and course designers worked collaboratively to develop a script introducing ratios and proportions. The primary aim of designing and delivering this math lesson in a video format was to elevate students' proficiency on multiple levels. Specifically, the lesson sought to enhance their procedural knowledge by guiding them through step-by-step approaches to solving proportion problems. Moreover, it aimed to deepen their conceptual understanding by clarifying the underlying principles and relationships involved in these problems. In addition, the lesson aimed to foster procedural flexibility, empowering students to employ a diverse array of problem-solving strategies in tackling proportion problems. Through this comprehensive approach, students were not only expected to achieve improved problem-solving and transfer skills but also to develop a robust foundation for tackling mathematical challenges with adaptability and confidence.

To implement this, the instructional videos were recorded as a live, semi-scripted lesson on proportional reasoning taught by a teacher in a diverse class of fifth- and sixth-grade students who were recruited for the recording. Building off of Begolli and Richland's (2016, 2018) video methodologies for research, we utilized 4K multi-camera video production with one camera focused on the teacher, the second camera captured the teacher and the whiteboards, and the third camera captured only the students. These camera angles allowed for post-production video editing manipulations, directing students’ attention to specific camera angles for increased engagement. During the recording process, we employed a reform-based instructional model in which the teacher first asks the students to solve a challenging proportional reasoning problem on their own, prior to receiving explicit instruction (Schwartz et al., 2011). Next, the teacher strategically chose students using the target solution strategies to describe their strategy to the class. Following each student, the teacher led a discussion on the procedures and higher-level conceptual overview of each respective target strategy: the equivalent fraction strategy and the unit ratio strategy.

4.5 Learning experience design

In cognizance of the established research surrounding the efficacy of user experience (UX) and instructional design (ID) within self-paced educational contexts, a deliberate approach was taken in the development of these online modules. Our design philosophy prioritized flexibility and learner-paced progression, affording students the autonomy to initiate their learning and proceed through the course at their own pace, in alignment with the principles outlined by Richardson et al. (2016). Furthermore, the design process included strategic decisions aimed at optimizing digital interactivity, mitigating mind-wandering tendencies, and enhancing learner engagement. Specifically, the original hour-long video lesson was divided into ten discrete segments instead of presenting it as a continuous uninterrupted stream. This segmentation was implemented to counteract the potential effects of mental fatigue and cognitive load while minimizing opportunities for students to drift into mind-wandering states, a concept that finds support from Mayer (2019). Additionally, the incorporation of scaffolded problem sets, in the form of worked examples, was meticulously integrated between the video segments. This design choice ensured that students had the opportunity to immediately practice problem-solving after acquiring new concepts through the video instruction. These types of problems, serving as scaffolds for novice learners, were strategically placed to highlight structural parallels within the lesson. This approach serves the purpose of directing students' attention towards key ideas, concepts, and relationships, thus facilitating their engagement with the material (Richland & Simms, 2015; Mesghina et al., 2024).

To illustrate, the learning process included a series of well-structured steps designed to promote active engagement and comprehensive understanding. Initially, students were prompted to recollect the strategies they had observed in the instructional videos. This encouraged them to reflect on the demonstrated approaches before moving forward. Subsequently, students were tasked with applying the procedural steps they had acquired to tackle a math problem. The subsequent phase entailed a comparative analysis, where students matched their own procedural steps with model example strategies. To facilitate this, video animations showcasing the sequential execution of mathematical procedures, termed "worked examples," were thoughtfully incorporated (Mesghina et al., 2024). These visual representations served to enhance conceptual comprehension and encourage comparisons between different solution methodologies. This approach provided students with the valuable opportunity to solidify their grasp of diverse solution strategies before advancing to the subsequent problem (Mesghina et al., 2024; Richland et al., 2012).

Following the comparison of problem types and solution strategies, students were tasked with independently solving the math problems, thereby evaluating their conceptual grasp of the material. Notably, solution reflections were seamlessly integrated within each problem scaffold. These reflections prompted students to articulate, in their own words, the process by which they arrived at solutions using their chosen strategies. This design choice was meticulously crafted to engage students in productive metacognitive assessments, encouraging them to contemplate the "how and why" of their solutions. This deliberate engagement in self-assessment has been shown to foster a sense of ownership in learning, augment preparation for assessments, and promote thorough review and practice (Tullis & Benjamin, 2011).

4.6 User experience design

Meticulous attention was dedicated to crafting a course interface that prioritized an optimal user experience design. A range of strategies were implemented to achieve this objective, ensuring that learners could seamlessly navigate the online learning environment while accessing course materials with ease. For instance, strategic design elements were employed to emphasize course roadmaps, objectives, and navigation instructions. These were highlighted and enclosed in distinct boxes to enhance their visibility and accessibility for learners. This approach was aimed at minimizing confusion and promoting ease of use. Moreover, standardized vector icons were consistently integrated before each instructional component. This design choice offered a clear demarcation between general interface instructions and specific lesson-related directions. Consequently, learners were able to effortlessly distinguish between the two, aiding in their navigation process.

To further enhance user experience, a table of contents and a progression bar were thoughtfully added. These elements provided learners with a visual representation of their journey within the course, offering a sense of orientation and progression. Clear instructions were also provided for each video, detailing functions such as pausing, playing, and re-watching. These measures were adopted to simplify technical interactions and ensure learners could focus their attention on the task at hand. Furthermore, to grant learners increased autonomy, the ability to navigate backward within the course was enabled. This adjustment was made to accommodate instances where students might make errors or navigate through the course too swiftly. By incorporating this feature, the course structure was designed to be flexible, allowing learners to move freely through the material and correct any navigational missteps. Ultimately, these deliberate user experience design choices aimed to streamline the learning process, reduce potential visual and technical obstacles, and enable learners to concentrate on the content pertinent to their learning objectives.

Another noteworthy enhancement introduced to the course involved the incorporation of intentional breaks from the math problem sets. These interludes were characterized by the introduction of enjoyable and captivating pop-up questions accompanied by GIFs. These questions were designed to encourage students' spontaneous responses. Given that our math lesson was enveloped in the theme of cooking recipes, the GIFs and corresponding questions revolved around students' culinary endeavors at home. Additionally, these inquiries prompted students to contemplate how the proportional reasoning concepts they were learning could be practically applied in their day-to-day lives.

Retaining the thematic coherence of our lesson, these intermissions provided students with valuable opportunities to share their activities during shelter-in-place and to illustrate how the acquired math concepts could be extrapolated beyond the confines of academia. These iterations embodied a collection of design choices firmly grounded in Learning Experience Design (LXD) principles. Collectively, these choices were employed in tandem to collaboratively shape the development of the online-video math course. The overarching objective was to seamlessly fuse elements reminiscent of a physical classroom with the interactive attributes intrinsic to online learning environments. This design approach aimed to optimize students' engagement in the online setting while concurrently curbing the frequency of mind-wandering. These adaptations were swiftly incorporated through the iterative co-design process, culminating in the creation of a dynamic and engaging online math course.

4.7 Data analysis

SPSS was used to examine scale reliabilities, generating descriptive statistics, conducting analyses to address missing data, exploring correlations, and employing AMOS 26 for structural equation modeling. To validate the instruments utilized in this study, assessments of scale reliability were performed. This validation was achieved by confirming the Cronbach's alpha coefficients, with values exceeding ɑ = 0.70 considered as meeting the acceptable threshold for reliability, as established by Nunnally (1994). To analyze the measured variables, a two-step approach was employed. Initially, Likert questions were recoded, and subsequently, descriptive statistics were computed for the items linked to each instrument. To address missing data, Full Information Maximum Likelihood (FIML) method was chosen as the estimation strategy. This approach is suited for dealing with random missing data, maximizing the likelihood estimation based on the observed data on a case-wise basis, as outlined by Carter (2006).

Initially, the validation of assumptions related to sample size, multivariate normality, linearity, and multicollinearity were examined. The evaluation of linear associations among the various study variables was carried out through bivariate correlation analysis. Subsequently, SPSS AMOS was employed to perform a path analysis, employing the maximum likelihood estimation (MLE) method. MLE was selected due to its efficiency in handling normally distributed data, asymptotic normality properties, suitability for large sample sizes, and widespread acceptance in structure equation modeling. This analysis aimed to investigate the conjectured research model, as illustrated in Fig. 1. In greater detail, we utilized covariance-based path analysis, which is a component of structural equation modeling (SEM). This approach enabled us to explore both simple mediation and sequential mediation within a unified model, employing measured variables to address our research inquiries (Lleras, 2005). Various fit indices were computed to assess the adequacy of the model. These indices provide insights into how well the model fits the observed data. The goodness of fit test conformity, often represented by the Chi-square statistic, evaluates the extent to which the model's predicted values match the observed values. Additionally, indices such as the goodness of fit index (GFI), comparative fit index (CFI), and normed fit index (NFI) gauge the overall fit of the model by comparing it to a baseline model, offering a comparative perspective on model performance. Furthermore, the root mean square error of approximation (RMSEA) assesses the discrepancy between the model's implied covariance matrix and the actual observed covariance matrix. Calculating and analyzing these fit indices provides valuable insights into the appropriateness of the model in explaining the relationships between variables (Kline, 2012). As such, the study explored separate direct and indirect effects, with the goal of comprehending the elements that influence students' tendencies for mind-wandering and their behaviors related to online engagement.

Fig. 1
figure 1

Hypothesized model for SUPR-Q (user experience), situational interest, mind-wandering, and online engagement

Full size image

5 Results

Descriptive statistics and correlations for the measured data (Perceived Learning Engagement, Situational Interest, Mind-wandering, and SUPR-Q scores) are provided in Table 1. In addition, the reliability estimates (Cronbach’s alpha) of all the scales are shown in Table 1. All of the measures were reliable based on the widely accepted recommendation of a Cronbach’s alpha of 0.70 (Nunnally, 1994).

During the preliminary phase of path analysis, we assessed a fully saturated model, evaluating conformity indices and the direct impacts of user experience, situational interest, and mind-wandering on students' engagement in online learning. The χ2 conformity index of the model was significant χ2 (1, N = 303) = 4.419, p = 0.036. Demonstrating a high degree of conformity typically necessitates a p-value that is not statistically significant, although this criterion becomes particularly sensitive when dealing with large sample sizes (Hoyle, 1995). To address the impact of the substantial sample size on the chi-square statistic's significance, the Root Mean Square of Approximation (RMSEA) was computed. In the preliminary model, the RMSEA stood at 0.061, with values < 0.08 indicating a favorable fit. Values closer to 0 suggest an even more precise fit, as outlined by Kline (2015). Following the approach by Kline (2005), the Standardized Root Mean Square Residual (SRMR) was computed by determining the square root of the difference between the residuals of the sample covariance matrix and the hypothesized model. The resulting SRMR value was 0.028, which falls below the recommended cutoff of 0.08, indicating a satisfactory fit. Lastly, the GFI, CFI, TLI, and NFI were calculated to be 0.992, 0.992, 0.950, and 0.989, respectively. Index values of 0.95 or above indicate a satisfactory level of conformity (Schumacker & Lomax, 2004). By estimating the structural β coefficients of the model, we identified statistically significant pathways encompassing all measured variables. This finding rendered a model re-specification unnecessary, leading to the selection of the initial hypothesized model as the final model (Table 3).

Table 3 Fit statistics for the hypothesized and respecified structural model (N = 303)
Full size table

Following the assessment of model conformity indices, a subsequent examination was conducted on the direct effects. A summary of the effects can be found in Fig. 2. The standardized coefficients can be interpreted as a change in units of standard deviations. For example, a 1 unit change in SUPR-Q is related to a 0.193 increase in standard deviations for online engagement. Consistent with our predictions, we found that both students’ SUPR-Q (user experience) (β = 0.193, p < 0.01) and students’ situational interest (β = 0.558, p < 0.01) had a significant positive direct effect on students’ online engagement. Furthermore, students’ mind-wandering (β = -0.088, p < 0.05) had a significant negative direct effect on students’ online engagement. Altogether, students’ user experience, situational interest, and mind-wandering together accounted for 51.7% of the explained variance in students’ online engagement (R2 = 0.517). We also found that students’ situational interest (β =—0.136, p < 0.01) had a significant negative direct effect on students’ mind-wandering, accounting for 1.9% of the explained variance in students’ mind-wandering (R2 = 0.019). Lastly, we found that students’ SUPR-Q (β = 0.674, p < 0.01) had a significant positive direct effect on students’ situational interest with it accounting for 45.5% of the explained variance (R2 = 0.455) (See Table 4).

Fig. 2
figure 2

Standardized structural regression model for SUPR-Q (quality of user experience on website), situational interest, mind-wandering, and online engagement for sample (n = 303). Note. **.Correlation is significant at the 0.01 level (2-tailed), *. Correlation is significant at the 0.05 level (2-tailed). Dotted lines represent non-significant path

Full size image
Table 4 Effect decomposition for the respecified model (N = 303)
Full size table

With the significant direct effects established, we proceeded to examine the potential mediating effects of students' situational interest and mind-wandering. To assess these mediations, we began by evaluating the significance of simple (mediated) indirect effects. These effects were examined in two contexts: firstly, between students' situational interest and their SUPR-Q scores, as well as their mind-wandering, and secondly, between students' situational interest and their SUPR-Q scores, along with their online engagement. Subsequently, we investigated the importance of the indirect impacts of students' mind-wandering on both their situational interests and their online engagement. The standardized indirect (mediated) effect of students’ SUPR-Q on students’ mind-wandering was (β = -0.092, p < 0.05). In addition, the standardized indirect effect of SUPRQ mediated by situational interest on students’ online engagement was (β = 0.384, p < 0.01). Next, we evaluated the significance of the simple mediation effect of mind-wandering on situational interest and students’ online engagement. The standardized indirect (mediated) effect of situational interest on online engagement was (β = 0.012, p < 0.05).

Lastly, we examined the cascading relationship of students’ user experience indirectly related to students’ online engagement through the sequential mediating effects of situational interest and mind-wandering. The standardized direct effect of students' SUPR-Q on SI (β = 0.674, p < 0.05), SI on MW (β = -0.136, p < 0.05), and MW on Engagement (β = -0.088, p < 0.05) were all significant. As a result, the standardized indirect (mediated) effect of students’ SUPR-Q on students’ online engagement was (β = 0.008, p < 0.05). The standardized total effect of SUPR-Q on SI was (β = 0.674). The standardized total effect of SI on MW was (β = -0.136). The standardized total (direct and indirect) effect of MW on PLE was (β = -0.088). The standardized total effect of SI on PLE was (β = 0.570). The standardized total (direct and indirect) effect of SUPR-Q on MW was (β = -0.092). The standardized total effect of SUPR-Q on PLE was (β = 0.577). These results suggest the partial mediation of the association between students’ user experience and online engagement through situational interest and students' frequency to mind-wander (See Fig. 2).

6 Discussion

Amid this unique time in education, designers of learning experiences encounter a twofold challenge. Their task extends to not only crafting learning environments that amplify conceptual comprehension by integrating theories from the learning sciences, but also discovering the means to generate captivating, human-centered experiences that provide support to a wide spectrum of learners. In light of this, we collaborated directly with teachers to co-design online video lessons that align with classroom curricula and cater to the needs of both teachers and students in a distance learning setting. This collaborative effort resulted in the co-development of a video-based online CCSS-aligned math course aimed at exploring the most effective methods to support students' engagement through the implementation of the LXD paradigm. As a result, the current study aimed to investigate the intricate relationships between perceived learning engagement (PLE), situational interest (SI), mind-wandering (MW), and user experience (SUPR-Q) in the context of an online math course. The discussion will focus on the implications of the findings, the alignment with theoretical frameworks, and potential avenues for future research.

Interestingly, our analyses demonstrated that students' user experiences and situational interest were positively associated with online engagement, while mind-wandering exhibited a negative association. These findings are consistent with previous research that emphasized the importance of positive user experiences and situational interest in promoting engagement (Chen et al., 2001; Hu, 2008; Simunich et al., 2015). The negative impact of mind-wandering on engagement highlights the need to address cognitive distractions and maintain focus during online learning, in line with the theory of mind-wandering as off-task processing (Smallwood & Schooler, 2015; Wong et al., 2023a, b). The mediation analysis unveiled that user experience indirectly influences online engagement through the mediation of situational interest and mind-wandering. This cascade effect highlights the complexity of the relationships and suggests that the quality of user experience can impact various facets of student engagement. Additionally, situational interest was found to mediate the relationship between user experience and mind-wandering, further underscoring its role in reducing cognitive distractions.

These findings resonate with the theoretical frameworks employed in the study, such as Learning Experience Design and Situated Cognition Theory, emphasizing the importance of interface design, learning design, and the online context in shaping students' engagement and learning behaviors (Brown et al., 1989; Mayer, 2019; Weigel, 2015). Additionally, the strong positive relationship between user experience and situational interest aligns with the notion that engaging and user-friendly interfaces can stimulate interest and motivation (Wong & Hughes, 2023). As such, this study emphasizes the critical role of LX designers in developing meaningful and engaging elementary mathematical courses, building off prior research from Begolli and Richland's (2018) cognitive experimental methodologies by expanding their video-based classroom mathematical teaching paradigm to different modalities such as online and blended learning experiences. Thus, by leveraging instructional design and human-centered approaches, this study shows that we can enhance students' situational interest and user experience, fostering increased engagement and overall learning outcomes.

These findings also illuminate our understanding of the application of LXD, suggesting a potential underlying mechanism explaining our LX design impacts on learners’ course experience and engagement. On average, we documented that students who experienced high user experience within the learning environment, showed greater situational interest within the math course, reduced mind-wandering, and ultimately showed greater instances of course engagement. We attribute these patterns to the learning experience design employed. Maintaining the ecological authenticity of the classroom setting within the online instructional videos enabled students to seamlessly track the teacher's and fellow students' actions, facilitating their immediate practice of mathematical concepts (Begolli & Richland, 2018). Additionally, the use of multiple strategy solutions, analogical comparisons, and worked examples may have further crystalized students' understandings (Kalra & Richland, 2022; Mesghina et al., 2024). These factors in the course design are likely to increase students’ situational interest and reduce students’ mind-wandering, as it provides a similar in-person learning context of the classroom while also leveraging the key affordances of asynchronous online learning (Begolli & Richland, 2018; Wong & Hughes, 2023; Wong et al., 2023a, b).

Careful attention was given to enhancing the course's usability with the aim of fostering a high-quality user experience design. This was achieved by prioritizing ease of use, findability, and navigability, aligning with recommendations from Simunich et al. (2015). To ensure maximum online engagement, we incorporated knowledge checks, scaffolded problem sets, math animations (in the form of worked examples), and math reflections that we intentionally sequenced between videos. These design decisions were strategically embedded within each math problem, guiding students through the learning process with sustained participation and interactivity. These user experience design choices offered students a chance to assess their initial comprehension of mathematical concepts and provided the flexibility to review and practice by rewinding videos or navigating back. This approach aligns with Wong and Hughes (2023), empowering students to actively regulate their learning pace and tailor their engagement to minimize instances of mind-wandering. Students spend less time attempting to navigate the course and are more focused on the learning experience as a whole. Therefore, the extent of interaction between learners and the online learning platform, along with a robust user experience, were likely crucial factors in fostering student engagement within the online learning environment. This approach demonstrated its viability in generating interest, mitigating mind-wandering tendencies, and ultimately increasing engagement, particularly when applied to young students learning mathematics online. As a result, the LXD paradigm showcased its potential to enhance the overall math learning experience and foster sustained engagement while learning online.

6.1 Constraints and limitations

Further research is needed to thoroughly assess the limitations and benefits of online learning experiences for elementary students. In our study design, we chose a self-paced course structure, allowing students ample time to complete each question. While this approach provided valuable data, some students spent excessive time on the math lesson, which may have impacted their learning in other subject areas. This survey analysis represents the initial step in a multi-year project to identify motivational and design factors affecting student engagement. Future research will include a log analysis for more precise engagement measures like time on task, course participation rates, assignments, and math accuracy. We plan to conduct a quasi-experimental study comparing students' situational interest, user experience, and engagement in a fully online versus blended learning course. Future studies will adopt a multi-modal approach, combining self-report data, open-ended questions, and clickstream learning analytics for comprehensive reporting. Although our specific study didn't explore treatment conditions between student groups, it is part of a broader research program investigating how relational reasoning and teaching with analogies enhance children’s math outcomes, and promotes higher-order thinking. This ongoing research aims to contribute to the field of learning experience design and its impact on student learning outcomes with educational technologies.

6.2 Conclusion

In summary, this study illuminates the complex interrelationship among user experience, situational interest, mind-wandering, and online engagement within an online math course. It underscores the significance of the theoretical foundations and practical ramifications for design choices in enhancing children's motivation, cognition, and behavior in the context of online education.

Given the differentiation between emergency remote online learning and the firmly established foundations characterizing evidence-based pedagogical online courses, this study presents an unique prospect for scholarly investigation. It specifically offers an exceptional avenue to engage in rigorous research endeavors, with a primary focus on elucidating strategies through which instructors can thoughtfully enhance their facilitation of support for students' learning experiences within the realm of online learning modalities. Results revealed that students' engagement is influenced by their user experience on the learning platform, as well as situational interests and mind-wandering. These findings highlight the importance of considering Learning Experience Design (LXD) principles to positively impact students' mind-wandering and engagement while learning online. Additionally, this research contributes to clarifying learning theories with educational technologies and provides specific design factors that explain how increases in students' user experience and situational interest can lead to reduced mind-wandering and improved online engagement. Moreover, it enhances our understanding of the mechanistic relationships inherent in students' learning experiences during emergency remote distance learning. This lays the groundwork for future inquiries that seek to incorporate Learning Experience Design (LXD) principles into the educational landscape now that students have returned to synchronous classrooms, but this also informs preparations in the event of such future disruptions. As the educational paradigm continues to evolve, this study provides valuable perspectives on LXD as a viable pedagogical framework, offering design elements that can be readily employed across diverse learning modes to promptly bolster elementary mathematics education.