Simulation-Based Learning and Argumentation to Promote Informed Design Decision-Making Processes within a First-Year Engineering Technology Course

Abstract

Analyzing the integration of scientific knowledge to guide decision-making processes supporting design challenges in engineering education is critical. However, effectively engaging in informed design decision-making processes is challenging, particularly in the context of online education. Simulation-based learning can bring authentic design practices to online education, but effective guidance and scaffolding must be provided to learners. Therefore, this research investigates the implications of integrating simulation-based learning with an argumentation framework to guide students in incorporating scientific knowledge into their design decisions, particularly in designing energy-efficient housing. This study took place during online learning due to the COVID-19 pandemic and was implemented within a first-year engineering technology undergraduate course. It aimed to analyze students’ decision-making processes when designing a zero-energy home for a Midwestern city using Aladdin, an integrated CAD/CAE platform that can be used to design a structure and simulate its function within a single system. This study investigates how students informed their decision-making processes in design for energy-efficient homes and the recurring trends in students’ designs related to economic decision making and energy science. The overall results show how cost constraints significantly influenced students’ observation and argumentation processes during their design challenge, highlighting the pivotal role of economic considerations in shaping their decision making. Moreover, the findings underscore the importance of holistic approaches in providing insights into teaching strategies for online learning, particularly in navigating the intersection of scientific and economic factors in design challenges.

1. Introduction

Providing students with authentic engineering design projects is challenging given its complex nature, as it often involves posing problems without clear-cut solutions [1]. In fact, developing structured and systemic thinking skills in this field poses various comprehensive educational challenges that require a holistic approach [2]. This involves addressing various interconnected aspects to cultivate a well-rounded understanding and application of structured and systemic thinking in the field [3,4]. For instance, in the context of engineering design for energy-efficient homes, blending scientific knowledge with economic considerations is essential to support design decision-making processes [5]. Also, design decision making must manage trade-offs such as overall cost, degree of safety, and various performance indicators like efficiency [6]. Furthermore, these engineering design tasks are difficult to implement in in-person teaching settings, let alone in online learning environments.

Simulation-based learning offers opportunities for teaching and learning complex and dynamic systems in engineering education [7,8,9]. For instance, simulation-based learning has been used in the context of engineering design to promote (a) design-thinking strategies [10], (b) science learning [11], and economic decision-making outcomes [5]. Such environments provide conditions that allow in-person and online learners to acquire design experience via interactive 3D simulations, which have resulted in substantial learning benefits [12,13,14]. The capabilities of these environments allow for the consideration of design constraints and engage learners in trade-off decisions informed by scientific principles [15,16]. However, given the complex nature of engineering design, students may often rely on trial-and-error methods rather than applying their scientific knowledge in project-based educational settings [14]. Thus, simulation-based learning needs to be accompanied by guidance to provide positive effects on learning [17,18,19].

This study contributes to the sustainable development of online learning practices and emerging technologies in education by proposing an educational approach to facilitate simulation-based learning for supporting the integration of economic- and science-informed decision making in the context of design practices. Specifically, this study implemented an argumentation framework [20,21] coupled with Aladdin, an integrated CAD/CAE platform that enables simulation-based learning, to guide students in utilizing their scientific knowledge to inform their trade-off decisions. Hence, through this research, insights into simulation-based engineering education are offered, enhancing strategies for teaching students to approach complex design challenges in energy-efficient housing. The research questions for this study are as follows: RQ1—How do students inform their decision-making processes for solving a design challenge in the context of energy-efficient homes? RQ2—What is the alignment between the claims, evidence, and reasoning underlying their decision-making processes? RQ3—What recurring trends can be observed in students’ solutions regarding economic decision making and energy science? By addressing these questions, we seek to deepen the understanding of how students engage in design trade-off processes via simulation-based learning and how they relate their scientific knowledge to their decision-making processes in an engineering project. Therefore, this research contributes to the sustainability education literature by examining how students integrate scientific knowledge into engineering design projects, focusing on the intersection of science-based and economic decision making in engineering design trade-offs.

2. Engineering Design Facilitated by Simulation-Based Learning

Simulation-based learning offers teaching and learning opportunities by (a) approximating practice, (b) allowing limitations of learning in real-life situations to be overcome, and (c) developing complex skills [17]. Simulation-based learning is an educational approach delivered via computer simulations, which are tools that reproduce the real-life characteristics of an event, situation, or system [22,23]. Specifically, in engineering education, simulation-based learning has been used to facilitate the learning of complex and dynamic systems and processes such as engineering design [7,9].

Engineering design thinking is a complex process, as it involves the realization of ideas in the form of devices, systems, or processes after careful cycles of scoping, generating, and evaluating such ideas [10,24]. Such ideas must function in order to achieve the client’s or user’s needs and must also specify given constraints [10]. In educational contexts, students must apply design strategies to engage in design thinking as they solve design challenges. These strategies can help learners understand challenges, build knowledge, and generate and represent ideas [18]. Other more repetitive strategies for optimizing designs involve weighing options, making decisions, revising design features, and performing experiments [18]. It is precisely in these iterative tasks that simulation-based learning can be particularly useful.

A specific type of simulation tool used in engineering design to support simulation-based learning provides affordances for computer-aided design (CAD) and computer-aided engineering (CAE). These tools enable rapid testing of designs, immediate feedback, and encourage evidence-based decision making for revisions [25]. Evidence-based decision making has been recognized as an essential strategy for improving the design process and problem solving [26]. Studies have shown that evidence-based decision making enhances design strategies, including creativity and knowledgeable decision making, by providing a step-by-step guide for planning, analyzing, optimizing, and validating designs [27]. Also, it addresses the high dropout rates in engineering by increasing student collaboration, motivation, and engagement.

However, as students engage in the ill-structured nature of design challenges, they often experience design fixation [8,23]. Fixation occurs when students, while addressing design challenges, do not make meaningful changes to the properties of the design artifacts. That is, fixation occurs when a designer becomes too attached to a particular aspect of the design and does not make changes, even when necessary, to improve it [28]. Thus, to overcome fixation, it is important to explore untapped avenues of creativity and gain a better understanding of where students struggle in the science and engineering learning process. A second challenge during design processes arises when students complete experiments using a trial-and-error approach and do not back up their design decisions with evidence or scientific principles when making trade-off decisions to meet the specified design criteria [29]. Also, students are often required to engage in argumentative discussions, and their ability to do so is effectively contingent on various factors, including their understanding of claims, evidence, reasoning, and the ability to construct and defend positions [20,30].

For decades, education researchers have investigated how to integrate computer simulations to maximize learning [31]. Approaches vary from teacher-led instruction involving an expository phase, where learners are first introduced to the required foundational knowledge, to other approaches that advocate for open student-centered approaches, where students engage in discovery processes [31]. No matter the approach, research has found that the role of guidance is relevant, either provided by the instructor or the simulation environment itself [32,33]. Meta-analytic works have identified that important considerations include the way information from the simulation is presented and integrated [33]. This suggests that a precursor or a companion to simulation-based learning should include preparing learners in terms of prior knowledge, disciplinary practices, and representational competence so that they can effectively interpret simulation outputs [34].

In the context of engineering practice, several studies have explored how educators and researchers have approached the problem of enhancing design education and learning in the context of design using argumentation as guidance [35,36]. Argumentation processes may promote collaboration by integrating scientific concepts and decision making based on evidence. Furthermore, they encourage thinking and enable students to use the skills they learn in various subjects, including many real-world examples. On the other hand, argumentation supports the development of reasoning, critical thinking, communication, and social behavior [30]. Various themes of argumentation, including incorporation, questioning, and uncertainty, have been explored in the literature to encourage diverse perspectives and critical examination [35]. It is challenging to analyze and discuss how students learn and, therefore, how teaching processes could be supported to enhance further educational outcomes. Hence, further research could be conducted on implementing technology in engineering-related argumentation by exploring how technology supports data collection and analysis and how students present their data [35,37]. Thus, educators play a pivotal role in facilitating argumentation among students, as they can introduce the concept of claims, evidence, and reasoning to students [20]. Providing insights into teaching strategies and incorporating language, literacy, and data into educational support materials could also improve the curriculum of a class [35,37]. Moreover, the literature has also identified note-taking as an effective tool to encourage active listening, information processing, critical thinking, and information retention [38], supporting students in organizing their thoughts and constructing coherent and persuasive arguments. Thus, the literature suggests that educators and researchers should continue to explore these approaches and adapt them to the evolving needs of students in engineering education, as further research and innovation are needed to enhance students’ argumentation and problem-solving skills in engineering.

3. Learning Design

An argumentation framework was used in this study to promote science-based decision-making processes as students approached a design challenge to build an energy-efficient house [20]. Based on the need for precursors or companions for simulation-based learning, argumentation was deemed adequate, as it guides learners in making predictions about optimizing their designs, applying their representational competence in interpreting the outputs of the simulation such as graphs and then justifying the outputs or results of their optimization processes grounded in some scientific concept or principle. Thus, our goal was to use an argumentation framework to guide students in decision making when approaching engineering design projects, together with their scientific knowledge. This framework comprises three components to support decision making: (1) claims, (2) evidence, and (3) reasoning. Particularly, reasoning is the process of explaining how evidence supports a claim. Therefore, claims and evidence are aligned as a prediction of a factor and its rationale (i.e., claim), which is validated by observing the hands-on results of the changes (i.e., evidence) [27,39]. In this context, students were guided to make trade-off decisions to solve the design challenge, enabling them to apply their scientific knowledge to inform their trade-off decisions. Specifically, as students made changes to optimize their energy-efficient homes based on the specified criteria, they were guided to focus on a specific design feature of their designed homes, predict the change that would result from modifying that feature (claim), interpret the resulting outcome of that change (evidence), and support the resulting change with a specific principle (reasoning) to document each of the iterations performed to optimize their designs. That is, students went back and forth between their design journals and the simulation, engaging in making predictions, making changes in the simulation, and then observing and justifying the outcomes of those predictions iteratively until their designs were optimized.

The argumentation framework was implemented in the context of simulation-based learning, where students used a CAD/CAE tool called Aladdin (version 1.0) to design an energy-efficient house. Aladdin is an integrated CAD/CAE platform that can be used to design a structure and simulate its function within a single system, enabling learners to engage in inquiry processes, enabling them to apply their scientific knowledge in the context of design [40].

As opposed to professional CAD/CAE tools (e.g., AutoCAD or SolidWorks) used by engineering practitioners in the workplace [17,19], Aladdin was specifically designed to support the integration of engineering design practices in educational settings. While practitioner tools may have significant learning curves, Aladdin was designed for novice learners, thus providing an intuitive graphical user interface. Accordingly, previous educational implementations have been conducted at the middle school, high school, and undergraduate levels (e.g., [11,25]).

Aladdin provides two types of learning engagement: (1) inquiry into sustainable building design and (b) inquiry into renewable energy design. The software utilizes computer graphics to visualize scientific concepts and facilitate the integrated learning and teaching of science and engineering (see Figure 1). It uses generative design to support decision-making processes regarding renewable energy design, inquiry-based learning, and AI-assisted design. Its graphical visualizations foster informed arguments, deepening the understanding of the interplay among variables in renewable energy design and, therefore, the impact of design choices on direct outcomes. Moreover, it provides visual feedback to facilitate inquiry practices in a design environment, allowing students to conduct experiments and analyze the results to make informed design decisions. Also, it can render multiple simulations simultaneously, enhancing inquiry-based design by enabling students to compare and analyze various data-driven arguments and decisions at once [41].

Figure 1 illustrates the previously discussed key features. For instance, the upper image shows the design canvas, construction’s localization effects, and associated design implications (e.g., temperature, sunshine, area, height, windows, and solar panels) in terms of cost and energy. The lower image depicts the visualizations and simulation capabilities for informed decision making, integrating data on net energy, cost, and specific details about solar panels, A/C, heaters, and geography.

The integration of the argumentation framework was used as support to actively promote and guide students in making thoughtful design decisions, resulting in reasoning through claims and evidence. The goal was for students to follow structured argumentation reasoning through this framework to simultaneously enhance their problem-solving skills, design skills, and scientific knowledge. Consequently, with every change in the design, followed by an execution of the simulation, students could support their claims about design changes (i.e., factor analysis) by providing evidence from testing and frequent reflection and analysis. Therefore, using Aladdin, students created designs for energy-efficient homes and documented their decision-making processes in a design journal, defining three factors related to the design:

• Claim—predicting what will happen after their design goes into effect;
• Evidence—observing the change;
• Reasoning—explaining whether the prediction aligns with the actual outcome.

These factors allowed the students to use the argumentative approach and effectively back up their arguments with reasoning and evidence to meet specific design constraints, such as energy efficiency, cost, size, and curb appeal.

4. Methods

4.1. Context and Participants

The study was implemented within a first-year undergraduate engineering technology course at a Midwestern university (N = 248) in the United States, covering foundational concepts in electricity, mathematics, mechanics, programming, basic statistics, and professional development. This is a required course for first-semester students and is one of the introductory courses of their major. The course was conducted during the fall of 2020 as a hybrid course, with all lectures recorded and uploaded to the course learning management system, and smaller recitation sessions held in person once per week. A weekly question and answer session was held synchronously through Microsoft Teams to review lecture content and example problems. These changes were in place due to COVID-19 restrictions and the large number of students in the course. The data were collected as part of the students’ engineering design challenge assignment and aligned with the following learning outcomes of the course:

• Identify and develop an academic pathway for success in a student’s selected major;
• Determine key elements and select appropriate strategies and technologies to solve technical problems;
• Apply computational tools to address technical problems;
• Define and demonstrate awareness of professional standards, practices, culture, and issues in engineering technology.

This course is part of the engineering technology program, which offers specific concentrations in automation and systems integration, mechatronics, robotics, computer engineering, electrical engineering, audio engineering, industrial engineering technology, supply chain management, and mechanical engineering technology. The students entering the course were expected to have a strong foundation in math, English, and lab science from high school, with eight semesters of math and English and six semesters of lab science. Moreover, the course intended to introduce foundational concepts in electricity, mathematics, mechanics, programming, basic statistics, and professional development. Its objectives included developing a conceptual understanding of heat transfer, solar radiation, heat flux (i.e., heat absorption), insulation, energy, and energy conversion concepts.

In parallel, the students were also required to enroll in a first-semester course in design thinking in technology. This course provided students with a foundation to apply engineering design thinking to engage in critical analysis of real-world problems and global challenges.

4.2. Procedures and Data Collection Method

The project design challenge was introduced in Weeks 8 and 9, which was halfway through a 16-week semester. The timing was intentional as it allowed students exposure to the project design process, where they were introduced to stages such as problem identification, research and requirements specification, concept generation, prototype construction, product integration and testing, and maintenance and updates. Students were also previously exposed to the engineering fundamentals of energy, electricity, and computer-aided design needed to complete the project within their course. In addition, prior to the completion of this project, students had successfully created three design-based models in Solidworks in the co-requisite lab portion of this specific course. The mid-semester timing also allowed for the completion of coursework in the co-requisite design-thinking course, where they were in the prototype and testing stages of their group project focused on design.

With regard to the specifics of the project, students were first taught the fundamentals of electricity and energy conversion, including how renewable energy sources work and the expected inputs and outputs. Solar energy was explained, including how solar panels function. After this, students were introduced to the project requirements and a short demo of the Aladdin software. Then, in each recitation session (smaller group setting of 50 min per week), instructors taught the students how to use the Aladdin software and focused on its design features. In addition, students were provided with recorded videos of the software components in the learning management system for review. This occurred for two weeks while students executed the project. The project comprised two major parts. The first part consisted of having students create a simple home to gain familiarity and practice with the design journal, the augmentation framework, and the Aladdin software. The second part consisted of solving a full design challenge. Students were given a week for each submission, and most started the work during the recitation session and completed it at home individually.

The data for this study consisted of a random sample of 50 design journals written by students who completed the design challenge of creating energy-efficient homes. This sample was used for the analysis. Accordingly, the data were analyzed by utilizing design journals where students documented their design decisions following the argumentation framework. Additionally, students were expected to demonstrate the application of design strategies to improve the qualities of their designs and apply economic analysis methods, such as cost-benefit analysis, to support their economic design decisions. The argumentation framework embedded in the journals guided students in predicting the outcome of a design feature, recording observations of the result after executing the simulation software, and explaining any disparities between the prediction and the actual outcome (refer to Figure 2). This structured approach aimed to enhance students’ argumentation processes during the design challenge as they optimized the features of their designed homes.

Figure 2 illustrates the structure of the design journal used to guide students in documenting their design processes. The guidance was provided in the form of indications and examples. The first column prompted students to identify the factor they planned to change in order to optimize their designs. In the second column, students were prompted to make predictions supported by evidence and reasoning. The third column elicited students to record the actual results after a change was made in the home design, accompanied by examples of evidence. The fourth column required students to explain their observations, supported by evidence and reasoning. These journal entries served as the basis for assessing students’ design decision making in solving the overall design challenge.

The design challenge prompted students to apply their knowledge of energy-related concepts to build an energy-efficient home using the Aladdin software. The assignment included restrictions on costs and size, and it had to be a net-zero home. The assignment was aligned with the learning objectives of the course, which included developing a conceptual understanding of energy-related concepts; applying design strategies to improve the energy efficiency of designs; and applying economic analysis methods, such as cost-benefit analysis, to advance economic design decisions. The data collection procedure involved written design journals to measure design decision making, which were submitted by students on the defined due date. As described before, the design journals comprised four columns, following the claims, evidence, and reasoning determined by the argumentation framework followed in this research:

• Factor: Description of the feature(s) of the design to change and how to proceed with the change;
• Prediction: Argument justifying why the factor was changed and the expected outcome(s);
• Observation: Description of the observed change in the Aladdin software as evidence;
• Justification: Reasoning process analyzing the rationale between the prediction and the observation

4.3. Data Analysis Methods

The data analysis was performed following a two-step procedure. First, two researchers collaboratively hand-coded the first ten observations, and then each researcher was assigned an additional 20 observations for hand-coding, resulting in around 50 initial observations for analysis. The first step in the analysis consisted of identifying students’ claims, evidence, and corresponding reasoning documented in their journals. The researchers then codified each of the three elements of an argument accordingly, and finally, they identified patterns in the data. With this approach, the analysis focused on how students guided their decision making in the design processes for energy-efficient homes and analyzed recurring trends. Specific analyses for approaching each of research question are explained as follows.

• RQ1: Scientific Knowledge in Engineering Design

The researchers conducted an open hand-coding process to identify and define related codes for each observation and factor, as represented in Figure 3.

The open hand-coding process consisted of assigning a descriptive label or phrase to each of the represented ideas. Then, the researchers labeled the codes based on direct quotes from the data and highlighted the claims in yellow, the evidence in aquamarine, and the reasoning in green. These highlight colors had no other meaning than to discriminate each phase of the Argumentation Framework. Each code was then categorized as a claim, evidence, or reasoning. Figure 3 illustrates an example of this preliminary process. Using the color legend (i.e., yellow for claims, aquamarine for evidence, and green for reasoning), each code was assigned a color corresponding to its category for each observation. To facilitate this, a matrix was employed with codes arranged in rows and observations in columns. Following this, the researchers calculated the frequency for each code and color category (see Appendix A, Appendix B, Appendix C and Appendix D). The codes and categories were then visualized using a spider chart. After completing the matrices for the three components of the argumentation framework and analyzing the frequency of each code, the researchers conducted an additional codification process, where each individual code was assigned to a particular category or topic, providing a descriptive overview of all codes within the same category. This secondary coding process resulted in six categories that encompassed the overall codes from the hand-coding process:

• Energy efficiency and conservation (EEC);
• Insulation and thermal regulation (ITR);
• Material selection and construction (MSC);
• Geographical and environmental considerations (GEO);
• Construction and space efficiency (CSE);
• Economic considerations (ECO).

The researchers then proceeded to associate each code with emerging categories based on previous numbers by counting the number of observations for each code and each new category. During this process, some quotes were selected to exemplify the categories and their corresponding coding. Visualizations were also created to show the patterns of each individual student’s arguments, with colors representing each of the six categories identified above as well as each identified claim, evidence, and reasoning.

• RQ2: Alignment between Claims, Evidence, and Reasoning

From the resulting information, the researchers generated some visualizations to address the second research question regarding the alignment between the three components of the argumentation framework. They created a spider chart to display multivariate data (i.e., the initial emerging codes and their assigned categories). The frequencies of the codes were visualized using lines radiating from a central point, forming a shape that allowed us to compare the frequencies across the three components of the arguments. The goal was to highlight the intrinsic relationship between the overall claims and evidence, supported by reasoning, thus depicting the alignment between an argument, its rationale, and practical evidence through the Aladdin software (see Section 5).

• RQ3: Pattern Analysis in Engineering Design Decision Making

After the coding process was completed, the researchers further compared and contrasted the six identified categories. Through this process, two major topics were identified:

• Economic design decision making;
• Energy science knowledge.

The researchers then utilized two visual representations, including a Sankey chart and a stacked bar chart, to visualize the overall patterns in the data. By using a Sankey chart, it was possible to analyze the flow between the six emerging codes from the first open coding process and the two emerging categories from the second categorization process. For this, the researchers took advantage of the codes’ frequencies, interpreting them as inflow and outflow distributions along with their rationale codes. Moreover, by using a stacked bar chart, they could support pattern analysis by identifying the proportions between the codes for (1) economic design decision making and (2) energy science knowledge for the three components of the argumentation framework (see Section 5).

4.4. Trustworthiness, Validity, and Reliability

For this study, interrater reliability was considered, indicating the consistency and agreement between the two researchers who executed the hand-coding, codification, and categorization processes. The researchers considered the same coding protocol previously defined and described for these processes. They used the consensus coding method, where they independently coded the data and then met to discuss the codes and decide on the final application of the coding scheme for the entire dataset. The two researchers worked together through regular weekly meetings for almost ten weeks to discuss the codification process and preliminary results to ensure that the coding process was consistent and reliable. During this process, the two researchers verified that consistency in coding was maintained, ensuring that the understanding of the coding process and criteria was clear. Moreover, the two researchers identified and addressed discrepancies in the coding interpretations from the first codification approach in a small dataset of observations. Also, through these regular meetings, the researchers engaged themselves in reflexivity to enhance the trustworthiness of this study by analyzing and reflecting on the impact of potential biases in the codification process.

5. Results

The following subsections present the results for each research question regarding (a) how students informed their decision-making processes for designing energy-efficient homes (see Section 5.1), (b) the level of alignment between students’ posed claims, evidence, and reasoning (see Section 5.2), and (c) the recurring trends in students’ designs in terms of economic decision making and energy science (see Section 5.3).

5.1. RQ1: Scientific Knowledge in Engineering Design

In general, students utilized six different science topics to inform their design decisions. The topics were (1) energy efficiency and conservation, (2) insulation and thermal regulation, (3) material selection and construction, (4) geographical and environmental considerations, (5) construction and space efficiency, and (6) economic considerations. For each of these topics, we present the specifics of the concepts and two representative quotes showcasing how students used them in their designs.

5.1.1. Energy Efficiency and Conservation

Students focused on implementing design features and technologies to minimize energy consumption and maximize energy conservation. This involved selecting energy-efficient systems and practices to achieve the goal of a zero-energy home. Figure 4 presents some quotes that exemplify this approach.

After analyzing the individual arguments of each of the 50 observations, a visualization was generated to further examine patterns in their argumentation process. In the context of “energy efficiency and conservation (EEC)”, 24 students centered their claims on this topic, represented with yellow. Figure 5 additionally reveals a pattern showing that eight students combined their claims considering aspects of “construction and space efficiency” (CSE), represented with red. For instance, within this group, one student contended that “more windows on the southern wall decrease the energy usage by passively heating the house, thus reducing the need for heaters”, and another student stated that “solar panels that are slanted receive the most energy from the sun, contributing to energy efficiency”. These approaches prioritized energy efficiency through strategic window placement, maximizing natural light and heat gain. Simultaneously, they optimized energy generation with slanted solar panel installation, efficiently utilizing roof space for renewable energy production and demonstrating a commitment to sustainable construction practices.

Also, Figure 5 reveals that the reasoning aligned much more with the stipulated claims than with the evidence utilized as support. In fact, despite the reasoning for the two aforementioned claims indicating that “more windows during the summer heated home too much that AC is being used more, thus drastically increasing the energy usage” and “if the house is flat, then we could easily install pole mounted panels to receive the most solar energy […] the placement of the solar panel being slanted helps receive the most energy for the house”, the evidence provided focused more on economic factors. For example, one student stated that “it allows people to have trees around their homes to look nicer and it also lowers the annual energy cost of the home," whereas other students only stated that the “annual energy cost is reduced”.

5.1.2. Material Selection and Construction

Students considered choosing sustainable and energy-efficient materials and construction techniques that minimize energy consumption. Figure 6 presents some quotes that exemplify this approach.

Figure 7 depicts a visualization of eleven students who focused on “material selection and construction” (MSC) in their claims, represented with blue. Among them, five combined their claims with “insulation and thermal regulation” (ITR), represented with green. For example, one student mentioned that “double-panned windows facing south generate heat energy from passive solar heating, utilizing sustainable materials and construction techniques to minimize energy consumption”, whereas another highlighted that “adding solar panels on the south side increases energy efficiency through material selection and construction techniques that maximize energy conservation”. These quotes illustrate students’ considerations regarding the integration of sustainable materials in construction to enhance energy efficiency, aligning with the goal of achieving a zero-energy home through insulation and thermal regulation in sustainable practices.

Nevertheless, Figure 7 shows a similar pattern to the analysis presented above in Section 5.1.1 regarding the evidence and reasoning students provided for their claims. In fact, evidence such as “double-panned windows facing south reduced annual energy costs” and “placing solar panels on the south side lowered electric costs” were used to support these claims, emphasizing economic considerations. However, the reasoning closely aligned with the initial claims, focusing on the effectiveness of passive solar heating by utilizing sunlight directly. For instance, one student stated that “passive solar heating is using sunlight to heat buildings directly […]; most solar energy when placed facing south; […] the annual energy cost of the home has been decreased”. Also, another student argued that “placing the solar panels south made the panels generate more energy from the sun because they got more sunlight facing the sun to generate more energy”.

5.1.3. Insulation and Thermal Regulation

Students focused on implementing design features and technologies to minimize energy consumption and maximize energy conservation. This involved selecting energy-efficient systems and practices to achieve the goal of a zero-energy home. Figure 8 presents some quotes that exemplify this approach.

The individual arguments for this specific claim were also grouped for analysis using a visualization. As shown in Figure 9, nine observations focused on “insulation and thermal regulation” (ITR). Among them, five combined their claims with “material selection and construction” (MSC), as discussed in Section 5.1.2, and three combined their claims with “geographical and environmental considerations” (GEO), represented with orange. For instance, students focusing their claims on this topic made statements such as “adding more windows at the south side of the house could allows sunlight to heat buildings directly, because in the Northern Hemisphere, south-facing windows receive the most solar energy” or “Indianapolis is on the North Hemisphere. Thus, the solar radiation will always come from the south”. In fact, students in this category acknowledged that optimizing south-facing windows enhances passive solar heating, improving insulation and thermal regulation to align with Indianapolis’s solar orientation in the Northern Hemisphere, where solar radiation predominantly comes from the south.

Again, this subgroup generally tended to align their claims with their reasoning, but these were misaligned with the evidence they provided, guiding their trade-off decisions toward non-scientific considerations, such as economic constraints (see Figure 9). To illustrate, the two students whose claims were provided above asserted for their evidence that “windows facing the south side of the house would cause the annual energy cost decrease as it won’t require the use of AC control its temperature” and “after the changing, the solar energy increased, and the net energy reduced, making the energy cost decrease”, respectively. Hence, despite considering scientific principles (e.g., geographic and environmental considerations, and insulation and thermal regulation), students prioritized economic sustainability, emphasizing viability as a factor that guided their trade-off decisions.

5.1.4. Geographical and Environmental Considerations

Students considered geographical and environmental factors such as climate and solar orientation, among others, as influential in optimizing energy generation and utilization within the specific context of a Midwestern city in the United States. They guided their design decisions to align with the location’s characteristics. Figure 10 presents some quotes that exemplify this approach.

Beyond the four students integrating claims with “insulation and thermal regulation” (ITR), as discussed in Section 5.1.3, a distinct subgroup emerged that concentrated on “geographical and environmental considerations” (GEO) alongside “construction and space efficiency” (CSE). This subgroup comprised six students out of twelve who focused on the two combined topics (see Figure 11). In fact, one student in this subgroup asserted that “adding more windows at the south side of the house could allow sunlight to heat buildings directly, because, in the Northern Hemisphere, south-facing windows receive the most solar energy, converting the sunlight to the thermal energy”, whereas another claimed that “since I had a lot of trees on the east side, they were blocking some of the sun radiation from hitting the solar panels […] by removing them from the sides of the house, I created more exposure to the sun since the sun rises in the east and sets in the west”. These two observations emphasized strategically placing windows and solar panels on the south side of the house, reflecting how this optimizes energy generation and utilization while enhancing space efficiency through direct sunlight heating. This approach demonstrates a holistic approach to construction trade-offs and their relation to geographical and environmental considerations.

Figure 11 further exemplifies the pattern discussed in earlier sections. The aforementioned student subgroup reinforced their arguments with evidence, stating that “more windows facing the south side of the house would cause the annual energy cost of the home to decrease” and “solar panels facing the south caused the annual energy cost of the house to decrease”, emphasizing economic perspectives. However, the rationale for these observations closely aligned with arguments from the claim phase, with arguments such as “in the Northern Hemisphere, south-facing windows receive the most solar energy, which can be converted into thermal energy, leading to a decrease in energy consumption” and linking to construction facilities by reflecting on how “solar panels facing the south generate more energy because they absorb more sunlight”.

5.1.5. Construction and Space Efficiency

Students focused on maximizing functionality while minimizing energy consumption through thoughtful design, including the choice of house placement, building form, organization and planning, building envelope, choice of building material, landscape design, and utilization of renewable energy. That is, students focused on the planning and configuration of the house, ensuring a holistic evaluation of the spatial layout and interconnected elements for its construction, contributing to the overall efficiency and sustainability of the house. Figure 12 presents some quotes that exemplify this approach.

Figure 13 illustrates insights from twenty students on this topic, in addition to the six previously discussed for “geographical and environmental considerations” (GEO) and construction and space efficiency” (CSE). Additionally, eight students, as discussed in Section 5.1.1, combined claims related to “energy efficiency and conservation" (EEC) and construction and space efficiency” (CSE). Students in this subgroup consistently based their arguments on “economic considerations" (ECO). For example, one student noted that “lowering the thermostat in winter months caused the annual energy of the house to decrease”, while another argued that “lowering the height of the building makes it easier to control the temperature inside the house because the volume of the house is smaller, significantly decreasing the net energy and energy expenses”. These arguments, supported by evidence, collectively emphasize a cost-effective approach to temperature control and energy savings.

5.1.6. Economic Considerations

Students considered and weighed economic factors, including initial costs, operational expenses, and long-term savings. Given some economic constraints, they considered cost-effective strategies and technologies to achieve energy efficiency while maintaining economic viability. Figure 14 presents some quotes that exemplify this approach.

Consistent with the preceding analysis, some students supported their claims with “economic considerations” (ECO), aligning coherently with the argumentation framework (see Figure 15). In this subgroup, three out of four observations aligned claims, evidence, and reasoning. In fact, one student stated that “changing the roof type to a shed roof can reduce the annual energy cost of the home”, providing evidence that “more solar panels can be placed facing south”, and reasoning that “the shape of a shed roof enables more solar panels to be oriented towards the south, maximizing solar energy absorption, leading to reduced energy costs”. Hence, despite incorporating scientific principles, students consistently emphasized economic sustainability in designing cost-effective solutions, showcasing a cohesive alignment between claims and reasoning throughout the analysis.

5.2. RQ2: Alignment between Claims, Evidence, and Reasoning

The second question in our study was related to identifying whether the focus of the concept informing students’ design decisions was consistent across their predictions and resulting arguments in terms of claims, evidence, and reasoning. For this, we performed a frequency count of the three components of student-generated arguments, which is shown in Figure 16. This visualization displays the distribution of the topics or concepts students used to inform their design decisions. The visualization also depicts the frequencies of the overall arguments in terms of claims, evidence, and reasoning for each of the topics, where yellow represents the claims, blue represents the evidence, and red represents the reasoning.

From Figure 16, it can be observed that there was a higher number of design decisions based on temperature regulation, shape or volumetric design, energy consumption and its relationship to solar heat gain through the involvement of solar panels, and the geography of the construction setting and its influence on the location or position of the facilities. This visualization also shows how the cost was influential with a higher ratio, guiding the students to inform their decision-making processes with this consideration.

In addition, the visualization provides information regarding the details of each specific topic and subtopic. For instance, the topic of economic considerations within the green box in Figure 16 had subtopics associated with energy cost, energy efficiency, improvement in temperature regulation, and so on. From the resulting plot, the alignment between the claims was analyzed. Predictions about design features supported by rationales were analyzed and corroborated through the Aladdin software to determine whether the prediction aligned with the actual outcome. That is, an intrinsic relationship between the claims, evidence, and reasoning, was expected.

From Figure 16, it can be observed that claims and reasoning align more closely with each other than with the evidence provided by the students. For instance, even when students incorporated scientific considerations into their claims, economic considerations influenced them more in making specific design decisions. This is evidenced by the closer alignment between claims, evidence, and reasoning for the topics of economic considerations and material selection and construction, both focusing on cost and savings. Several quotes show how students related their argumentation process to “cost” considerations (refer to Figure 17).

Finally, the visualization in Figure 16 also illustrates students’ arguments and their relation to the emerging categories from the subtopics. Each category is represented by a colored square: “energy efficiency and conservation” (black), “insulation and thermal regulation” (blue), material selection and construction (yellow), environmental considerations (purple), construction and space efficiency (red), and economic considerations (green). The figure also shows how students’ claims may be aligned with more than one emergent category, represented by corresponding colored squares for each code. For instance, in the ‘material selection and construction’ category, the codes ‘kinetics’ (K), ‘number of solar panels’ (NSP), ‘wood might keep heat better’ (WHB), and ‘wooden heat retention’ (WHR) are identified with yellow squares.

Furthermore, ‘kinetics’ is shared with “construction and space efficiency”, where a student stated, “thick walls would store more heat energy in winter, as they have more space for molecules to collide”, linking decisions on construction (i.e., house planning) to their scientific perspective. On the other hand, NSP is shared with ‘environmental considerations’, with a student noting that “adding more solar panels at the south side of the house could allow solar panels to receive more solar energy”, linking solar panel numbers to geographical factors. Moreover, WHB and WHR are shared with “insulation and thermal regulation”, with a student arguing that “wood might keep heat better than metal”, emphasizing wood’s thermal properties for reducing energy consumption.

5.3. RQ3: Pattern Analysis in Engineering Design Decision Making

To address the third research question of this study, which focuses on identifying the recurring trends observed in students’ designs in terms of economic decision making and energy science, a Sankey chart was utilized (see Figure 18).

The Sankey chart in Figure 18 depicts the inflow and outflow distributions between the two categories: one aligned with economic design decision making and the other aligned with energy science knowledge. Economic design decision making informed decisions related to construction and space efficiency, energy efficiency and conservation, geographical and environmental considerations, as well as insulation and thermal regulation. Energy science knowledge informed decisions related to energy efficiency and conservation, geographical and environmental considerations, installation and thermal regulation, as well as material selection and construction. In addition, a stacked bar chart (see Figure 19) depicts the relative proportions between the two categories in terms of the instances where they were used in students’ claims, evidence, and reasoning, with economic considerations being more frequent for informing design decisions. That is, economic design decision making had a greater influence than scientific knowledge in supporting students’ design decisions, as evidenced by their arguments focused on energy costs, expenses, and savings.

In our final analysis, two Sankey charts were generated to reveal the alignment between claims, evidence, and reasoning. Figure 20 depicts the alignment of economic design decision making and energy science knowledge in students’ designs. After a second coding process, the impact of economic considerations alongside scientific knowledge on students’ decision making can be identified. The plot also indicates that claims and reasoning are more aligned with each other than with evidence, pointing to a holistic challenge in fostering students to incorporate new evidence into their scientific explanations as part of their informed decision-making processes [39]. On the other hand, Figure 21 highlights students’ adherence to the argumentation framework, emphasizing the alignment between claims and reasoning. The prioritization of non-scientific considerations like cost-effectiveness, energy efficiency, and conservation suggests that students recognized the central role of these topics in sustainable design.

6. Discussion and Implications

The results of this study indicate the educational value of an argumentation framework coupled with simulation-based learning, as some students not only possessed a theoretical understanding of scientific principles but also actively applied this knowledge within the context of engineering design. Their incorporation of science into design claims signifies a nuanced and thorough approach to integrating scientific concepts into the broader framework of the engineering design process. It is important to note the factors students consider when formulating their design decision making. These factors, including energy efficiency and conservation, insulation and thermal regulation, material selection and construction methods, geographical and environmental considerations, and construction and space efficiency, collectively reveal a holistic comprehension of the various dimensions of energy-efficient housing.

The outcomes also evidenced how the students made decisions that balanced the ideal of energy efficiency with the practicalities of budgeting and economic feasibility, as Figure 16, Figure 18, and Figure 19 show. Specifically, the outcomes revealed that cost constraints significantly influenced students’ observation and argumentation processes during the design challenge. The limitations, particularly those related to cost, played a pivotal role in shaping how students approached gathering observations and forming arguments, thus evidencing trade-off decisions students had to make. The software may have highlighted economic considerations as students navigated the design solution process, influencing the students’ arguments. As a result, the students focused on collecting observations that were not only relevant to the technical aspects (i.e., scientific knowledge) of the design but also directly tied to the economic feasibility and cost-effectiveness of their proposed solutions, supporting their argumentative decision-making process. As a result, the alignment between claims, evidence, and reasoning varied depending on the factors considered. While claims and reasoning exhibited a higher degree of alignment, evidence, representing the observations of the stated claims, demonstrated greater alignment when economic considerations were also part of the analysis. This suggests that economic factors played a significant role in shaping the coherence of the argumentation framework.

Regarding the alignment between claims, evidence, and reasoning across the students’ arguments, the radar chart in Figure 16 indicates that the range between the three components of the argumentation framework is narrower when economic considerations are involved. Moreover, the plot presented in Figure 20 also shows how, among the claims, evidence, and reasoning, students tended to prioritize economic considerations over energy science knowledge in supporting their design decision-making processes. This suggests that economic considerations substantially impacted the alignment between claims, evidence, and reasoning more than the other factors considered in this study.

For instance, one student claimed that adding more windows on the south side of the house would decrease the home’s annual energy cost. The evidence provided to support this claim was the observed decrease in energy costs when windows faced the south side of the house. The reasoning behind this claim was leveraging natural sunlight to reduce energy costs. This example illustrates how students incorporated technical and economic considerations into their decision making, demonstrating a holistic approach addressing scientific principles and cost-related factors.

On the other hand, the Sankey chart presented in Figure 21 provides a visual representation of the alignment between the emerging codes of the axial coding process, specifically focusing on the relationship between claims, evidence, and reasoning in the context of engineering design decision making. It illustrates how claims were supported by evidence and reasoning within the context of economic decision making and energy science knowledge, as well as how the claims and reasoning processes involved more considerations compared to the evidence process, as seen in the number of inflows to each stage of the argumentation framework. Particularly, it is important to note the alignment between the claims and reasoning processes. Nevertheless, the evidence process is not as aligned, not only in the division between economic and scientific considerations but also in a general sense, where the overall consideration in the evidence process was significantly lower than that in the other two stages of the argumentation framework.

The findings from this study align with those from a previous work that consistently demonstrated the effectiveness of using an argumentation framework to support students’ use of scientific knowledge to inform their decision-making processes [42]. However, the study also demonstrated that in some instances, economic factors influenced students’ proper alignment of claims, evidence, and reasoning. Research in engineering education has emphasized that in the context of engineering design, arguments must go beyond the use of scientific principles to also consider economic, aesthetic, and ethical considerations, among others [35,43]. So, in this regard, the findings from this study show evidence supporting this notion. However, as evidenced by some observations, some students presented misaligned arguments. For instance, they made claims based on scientific principles but then provided reasoning that was aligned with economic or other considerations. Thus, the implementation of the argumentation framework could be revised so that students pay closer attention to this alignment. One strategy that can be implemented to help students better align their arguments and improve the overall quality of their arguments is to supplement them with metacognitive reflections [44]. Computer-based scaffolding can also be added to the simulators to prompt learners to state their arguments and automatically verify the alignment between claims, evidence, and reasoning [42,45,46,47].

The findings of this study also make a case for the use of simulation-based learning to support the implementation of engineering design learning activities for online learning [12,48]. However, in this regard, it is imperative that simulation-based learning is complemented with scaffolding [17,49], whether it be instructor-enabled, peer-enabled, or technology-enabled [50,51,52]. In the case of CAD/CAE simulators, artificial intelligence (AI) or machine learning (ML) capabilities can further support designers/learners in generating feasible solutions (e.g., [53]). For instance, more recently, Aladdin has been enhanced with AI capabilities based on generative design to explore the entire parameter space supported by the software to find optimal solutions iteratively [40].

7. Conclusions

The findings from this study contribute to the sustainable development of online learning for promoting engineering design practices enabled by emerging technologies integrating simulation-based learning with an argumentation framework. This approach supported students in engaging in engineering design processes for online learning during the COVID-19 pandemic. To validate the feasibility and effectiveness of this approach, this study analyzed the alignment between students’ claims, evidence, and reasoning in their design decision-making processes.

The results highlight the effectiveness of this approach in helping students move away from trial-and-error approaches to make informed decisions when engaged in design processes. Students based their decisions on scientific and economic considerations, thus being more comprehensive in their design decision-making processes. However, as a limitation of the approach of this study, it was identified that economic constraints influenced students’ observations and argumentation processes during the posed design challenge. This led to a misaligned decision-making process prioritizing economic considerations over scientific knowledge (i.e., scientific principles). Specifically, the results let us assert that while claims and reasoning exhibited a higher degree of alignment, the evidence, representing the observations of the stated claims, demonstrated greater alignment when economic considerations were also considered as part of their analysis. A second limitation of our study was the sample of 50 observations. Although the 50 observations are adequate for a qualitative study, our study went beyond the qualitative analysis that addressed research question one, aimed at identifying and characterizing the nature of students’ arguments, to a quantitative descriptive analysis where qualitative data were then quantized to describe patterns addressing research questions two and three. Therefore, as part of our future work, we will expand the analysis to the full sample, and in the process, we will investigate whether computational methods, such as those from natural language processing, could support the analysis procedures.

Despite its limitations, this study’s findings are promising, indicating that educational implications involve providing students with explicit training on the argumentation framework. This emphasizes aligning students’ design decision-making processes in terms of claims, evidence, and reasoning, thereby improving their design arguments.