The Impacts of Integrating Introductory Composition , Communication , and Design Thinking Courses

Much recent STEM research indicates that course integration improves the student learning experience and fosters stronger connections among concepts and skills; this study attempts to evaluate whether or not students learn the design process more fully in the integrated version of a required first-year course, Design Thinking in Technology. Drawing from an ongoing assessment of an Integrated First-Year Experience at Purdue University, this article reports on the challenges of teaching design thinking and analyzes whether students in an interdisciplinary course integration can demonstrate the work of their design processes more completely and effectively compared to students in a nonintegrated version of the course. We employ a modified version of the Engineering Design Process Portfolio Scoring Rubric (EDPPSR) as a method of evaluating students’ design portfolios. Our initial and follow-up analyses show that students in both versions of the course struggle to complete design journal assignments satisfactorily. We assess and analyze the impact of STEMhumanities integration on students’ abilities to document and contextualize the design process using journals, and also offer discussion and suggestions about our findings.

Design thinking has the potential to be an umbrella skill encompassing several other valued skills, such as creativity, critical thinking, innovation, empathy, collaboration, information literacy, and audience awareness.Educators and employers see these skills and abilities as crucial tools for the 21st century.The Partnership for 21st Century Skills Report indicates that employers place increasing importance on creativity and innovation (Casner-Lotto & Barrington, 2006).Creative thinking and critical thinking have long been common terms in conversations about what college graduates most need as they transition into the workforce; "design thinking" is a relatively new addition to such discussions."Design thinking" involves a strategic, practical process of conceiving and actualizing solutions to problems.Design thinking is not something only designers can engage in, and its process can become a powerful agent of change, especially when used in collaboration and with dialogue among multiple stakeholders (Brown, 2009).Tom and David Kelley (2013) also cite the strong potential for design thinking and empathy, creative thinking, and iterative approaches it encourages to make the world a better place.Critical design thinking skills and processes can be difficult to teach, especially in ways that adequately reflect the interdisciplinary nature of how those skills and processes are used in real-world businesses and industry.Finding opportunities to foster these skills is important for preparing students to recognize, value, and transfer design thinking across disciplines.
In the Polytechnic Institute at Purdue University, educators have introduced a STEM Integration model for their first-year gateway course, Design Thinking in Technology.This integration effort was motivated by a perceived need to more clearly demonstrate the value of critical communication in combination with design thinking, and to teach these skills in a holistic, connected, interdisciplinary context.Much recent STEM research (Bannerot, Kastor, & Ruchhoeft, 2010;Guthrie et al., 2012;Honey, Pearson, & Schweingruber, 2014;Kellam et al., 2013;Rhee et al., 2014) indicates that course integration has the potential to improve student learning.We hypothesize that an integrated program will help students recognize the importance of design and demonstrate this learning more concretely as a result of seeing both communication and composition principles at work within the design process, and vice versa.In this integrated model, instructors from the Polytechnic Institute join with instructors from the College of Liberal Arts to teach integrated sections of their courses and create an atmosphere where empathetic audience awareness, design thinking, and communication skills are valued and taught as cohesive, interlocking, iterative practices that students will need to succeed in their future lives and careers.
This study attempts to quantitatively evaluate whether or not students in the integrated version of the Design Thinking course learned to articulate the design process more fully.Using the Engineering Design Process Portfolio Scoring Rubric (EDPPSR) to analyze students' final design journals in both integrated and non-integrated sections, we measure whether the Integrated First-Year Experience had the intended effect on students' abilities to document and demonstrate their understanding and experience of a team-based design process.In the article that follows, we first review existing literature about both design and STEM course integration, then describe our data collection and analysis.We then offer further discussion points and exploration of our results, and finally push for future research and assessment of technology students' design abilities.

Literature Review
This course integration was developed and implemented specifically to demonstrate the interconnectedness of design thinking, critical problem-solving skills, and strong communication skills in both oral and written modes.Though design thinking and many other 21st-century skills are increasingly prized by employers, they can be difficult to teach and assess.The Integrated First-Year Experience described below seeks to address this difficulty and bring additional support to the challenges of teaching and learning design.
Many course integration programs in STEM fields are generally geared toward developing and increasing 21st-century competencies, fostering readiness for the STEM workforce, and generating student interest and engagement (Honey, Pearson, & Schweingruber, 2014).Wang, Moore, Roehrig, and Park (2011) explained, "STEM integration is a curricular approach that combines the concepts of STEM in an interdisciplinary teaching approach."A variety of integration programs have been discussed and studied in existing literature, many within STEM disciplines and some involving broader collaborations (See Bannerot, Kastor, & Ruchhoeft, 2010;Guthrie et al., 2012;Rhee et al., 2014).Explicit connections are commonly made in the contexts of engineering and technology, which are known for design activities (Grubbs & Strimel, 2016); such "technological and engineering contexts bring attention to the increasingly important role that STEM plays in our society and emphasize how STEM affects our everyday existence" (De La Paz, 2013).Often these integrations involve specialized capstone or "cornerstone" courses taught at either the beginning or end of a student's undergraduate career.Conversely, Kellam et al. (2013) described integration among design, engineering, and social science courses threaded through four years of their engineering program, reporting that the main goal of the program, "is for students to develop a deep understanding of the larger socio-technical systems in which engineering is situated" (p.8).They hope that "students will develop an understanding of the interrelationships between engineering, the social sciences, and the humanities" (p.9).
The goals of our Integrated First-Year Experience are similar.In creating an integrated, interdisciplinary course for teaching design alongside both introductory composition and communication skills, we are working to jointly foster opportunities for learning and practicing innovation.The tools and skills of the design process ideally come together in this integration with the tools and skills of communicating orally and in writing, drafting, revision, following conventions, thinking rhetorically, understanding audiences, conducting and citing research, and so on.As students practice using these skills and tools in concert, instructors from all three disciplines (design thinking, English and oral communications) involved are available and prepared to encourage and advise them.
Design and design skills are inherently difficult to teach, due to the unique epistemology of design-"we come to know through active and purposeful construction of new knowledge" (Rowland, 2004, p. 43) and only a small part of design knowledge can be readily shared.Several design theories describe an epistemology which requires that knowledge is constructed by experience.Knowledge that is learned through experience and constructed through continual practice can also be described as a tacit-knowledge or knowing-in-action (Schön, 1995).As Schön (1995) described, an expert who tries to teach their craft or practice must reflect on specific situations and contexts to describe how they would approach them.It is in this highly contextualized, individual manner that design knowledge is created, through reflection on the practice and the process.Schön described this as either reflection-in-action or reflection-on-action, and such reflection is crucial to design.Reflection is also important within the relationship between problems and solutions; well-designed solutions align with the problem as stated at the beginning of the process.Here, as well, the nature of design is contextualized and difficult to isolate.The iterative process of reflecting and aligning problem and solution gives credence to the concept of problem and solution co-evolution (Rittel & Webber, 1984).Essentially, when working with a complex design problem, also called a wicked problem, the designer is looking to define the problem in a specific context.The process of defining the problem, researching, reflecting, ideating, and reflecting builds an understanding of the context in which the problem is situated.
Designers, no matter their discipline, need to reflect-in-action and reflecton-action, define problems and solutions simultaneously, and organize their thoughts before acting.Experience in design education is intended to scaffold the adoption of such designerly ways of thinking (Cross, 1982).After helping novice designers to more fully understand the solution and the problem, reflection on the process further builds the designer's knowledge base.Then they can apply the principles learned from their experience to a new problem and context (Lawson & Dorst, 2009).Students and practitioners of design in any context should know how to organize their thoughts, document their process, and communicate both effectively.Why did they decide on this solution?Why did they brainstorm these alternatives?How did they arrive at this problem and context?Who are they communicating with and why?Designers must provide logical rationale for their decisions and evaluate themselves on the performance.In fact, using design journals to document and become conscious of the design process and answer questions about the actions of that process involves a reflective process that reciprocally reinforces learning (Lin, Hmelo, Kinzer, & Secules, 1999).The design journal assignment described and analyzed below was specifically meant to help students practice this important step of documenting design processes in preparation for communicating and justifying those process to others in a variety of contexts.

Integrated Instruction for Design Thinking
Design Thinking in Technology is a required, college-specific course for all majors in the Polytechnic Institute at Purdue University.In this course, students are expected to identify and think critically about a user's problem, choose and clearly define that design problem within the context of a global grand challenge, and research the implications of previous solutions.Students are also expected to synthesize multiple data sources to make informed design judgments.To provide evidence for their design process, students must be able to communicate in both an oral and written format.
Administrators and faculty within the Polytechnic Institute and the College of Liberal Arts developed an Integrated First-year Experience program aimed at connecting the curricula of three introductory courses: Design Thinking in Technology, Introductory Composition, and Fundamentals of Speech Communication.In the integrated versions of Design Thinking, half of the students in the course are concurrently enrolled in a Composition (English) course together, and half are enrolled in a Speech Communications course (Chesley, Mentzer, Jackson, Laux, & Renner, 2016).In this integrated version of the Design Thinking course, curricular connections to Composition and Communication courses were meant to support and foster holistic improvement in students' composition, writing, oral presentation, and critical design thinking skills.A student enrolled in this Integrated First-Year Experience during the Fall 2016 semester would share instructors from two of the three disciplines-either in Design Thinking and Composition or Design Thinking and Speech Communication.In addition, about one-half of the students in each section of Design Thinking were in the Composition course while the other half were in the Speech Communication course.Each Design Thinking course thus acts as a central point in a "trio" of integrated courses.
The partnerships among all three courses emphasize productive and symbiotic intersections between the humanities and STEM disciplines.Instructors and administrators from each subject, Technology, Communications, and Composition, collaborated to weave their curricula together and provide students with a variety of direct and indirect opportunities for making connections between Design Thinking and their humanities course.These opportunities, depending on individual instructors' implementation, included inclass activities focused on applying concepts of effective communication, assignments in one course drawing on content or topics covered in another, and shared teaching events where instructors joined each other's classrooms to discuss connections across their curricula.Table 1 outlines the substantive differences between a non-integrated and integrated Design Thinking course.A primary difference for all integrated sections of Design Thinking involved a modification of the final design project to directly include and draw on skills and concepts from all three disciplines-Composition, Communication, and Design.Various pieces of the final project were ultimately presented as a culmination of students' cross-disciplinary teamwork in all three courses.Many students composed detailed research papers, posters, websites, or videos in their Composition course on the same problems and solutions they worked with in Design Thinking.All teams in the Design Thinking course also prepared an oral presentation about their innovative design projects.To accompany their more formal design work, student teams also compiled a design journal documenting their process over the final half of the semester.The pieces of this final project offer several obvious points of assessment as to the impact of the integration.For this particular study, we focus on the design journals students completed concurrently with the design work of their final projects.Additional details about the assignment and its context are included in the next section.
Our hypothesis is that instructors' pedagogical efforts to integrate Design Thinking, Composition, and Communication would lead students to create stronger, more robust, more organized design journals, with clearer, more logical answers to questions about why, how, and how well they made the choices they did during the second half of the semester.Because students in the integrated sections of this course were simultaneously learning, either in a Communication course or a Composition course, about the most effective ways to arrange and present information to an audience, we expected evidence of this learning to make a difference in the quality of the design journal assignments students submitted.

Design Thinking and Design Journals
The final design journals utilized in this study were assigned as the culmination of the students' final project in Design Thinking.This final project asked each team to select a grand global challenge, identify a localized manifestation of the problem related to the challenge, and develop a solution via research, prototyping, and testing.Articulating the logical path of one's design decisions is an important step in successfully thinking and working like a designer.The final design journal assignment required a full documentation of the final project described above.Near the beginning of the final project's eightweek duration, students were introduced to the design journal assignment and instructed to track and save all their individual and group work (most of which is also turned in at intervals throughout the project).Throughout the project, students were asked to document and communicate their process and results, using a shared storage space or shared document that would eventually become a portfolio of their collaborative design process.
In the Team Design Journal assignment prompt students received prior to the beginning of the final project, students were asked to "keep a single team journal to which all members have access (i.e., it should be kept in a collaborative workspace).This journal will be used to document all work by all members of the team, which includes work performed both collaboratively and individually."In class as well as in assignment prompts, teams were encouraged to take advantage of Google Drive, Microsoft OneDrive, or Blackboard as shared workspaces where materials can be collected from and shared with all team members.In addition to the basic assignment prompt, students were also given a copyable Google Doc template with some further instruction and placeholders for all required elements of the design journal, from the beginning (Problem Definitions & Fieldwork Planning) to the end (Final Presentation Preparation materials).Prefacing these placeholders, a brief set of instructions tells students that "All your work should be entered in the design journal here," and "The name or description of the assignment should be first on the new page and be a heading of an appropriate level (notice a few have been built as examples to modify and follow)."The template also advises students to "Begin with the format suggested here, but be creative in telling your story.The purpose is to document your journey this semester with this journal."The journal was an ongoing collaborative assignment throughout the seven or eight weeks of each team's final project, ultimately submitted during the final weeks of the course.
The design journal portion of the final project was meant to be an overarching portfolio describing the design process of each team, submitted at the end of the semester to accompany their final project and presentation (Groves, Abts & Goldberg, 2014).Students' documentation of their final project design processes forms the basis of our artifact analysis.Each student team's collaborative journey from problem to solution is what their design journal deliverable should cover.This study looks at the design journal as a unit of analysis because it is the culmination of the students' thinking and design decisions over the eight weeks of this final design project.

Research Goals & Methods
Integrating the curriculum of our Design Thinking course with that of the Communications and Composition courses is specifically meant to help develop a stronger design thinking mindset in all first-year technology majors.Making explicit connections, thinking critically about problems and solutions, and communicating effectively are common objectives among all three disciplines.In assessing the impact of this integration on student's design abilities, we ask: did students in the integrated version of Design Thinking learn to more effectively document and communicate their design process as they completed their final project?
To explore whether this integration is improving student learning of design thinking, we collected and compared final design journals from students in both integrated sections and non-integrated sections of the course.The design journals were used as the best assessment method because they are "worthwhile activities that relate to [our] instructional outcomes and allow [our] students to demonstrate what they know and can do" (Perlman, 2003, p.3). Analyzing the design journals from students' work on their final projects should provide an indication of the students' design thinking mindset after the course instruction.

The Engineering Design Process Portfolio Scoring Rubric
The Engineering Design Process Portfolio Scoring Rubric (EDPPSR) is meant to "allow student performance in the underlying knowledge and skill areas to be reliably and repeatably [sic] rated" (Groves, Abts, & Goldberg, 2014, p. 24).The EDPPSR was originally developed as a tool for evaluating capstone engineering design project journals in K-12 settings, and the rubric is continually being tested and validated for reliability (Groves, Abts, & Goldberg, 2014).Although the rubric is still being refined, we selected the rubric for this study because the elements aligned with the project outcomes of the assignment artifacts we collected (Coots et al., 2017).This rubric will help us quantify evidence of students' design thinking mindsets as collected in the final design journals.
The EDPPSR covers 14 elements of the engineering design process, all identified by a collaborative research team throughout a decade-long development process through their collective engineering design experience and expertise in performance-based assessment (Groves, Abts, & Goldberg, 2014).Each element within the EDPPSR is evaluated at one of six scoring levels: 0 (no evidence), 1 (novice), 2 (developing), 3 (proficient), 4 (advanced), and 5 (exemplary).For example, for Element A, "Presentation and justification of the problem," a design journal received evaluation of 5 if "The problem is clearly and objectively identified and defined with considerable depth, and it is well elaborated with specific detail; the justification of the problem highlights the concerns of many primary stakeholders and is based on comprehensive, timely, and consistently credible sources; it offers consistently objective detail from which multiple measurable design requirements can be determined." Not all rubric elements were ultimately relevant for our application of the EDPPSR.After an initial review of the design journals, two elements in the rubric were deemed irrelevant for this particular artifact.Element E, the application of STEM principles and practices, was omitted from the evaluation because students had not been asked to evaluate their designs utilizing these principles.There was limited evidence in the final design journals that this element was a part of the course curriculum, and it was thus removed from the rubric.Element M, presentation of the project portfolio, was likewise omitted.While there was an in-class final presentation for the project, researchers were not evaluating the oral presentations but rather the written documentation.All other elements of the rubric were evaluated on a 0-5 scale as prescribed in the original rubric.
Each design journal was evaluated, and each element scored according to the EDPPSR.The EDPPSR was also used for grading and assessment in the Design Thinking course.However, researchers applied this rubric not to assess student effort for a grade, but to independently come to better understand the students' abilities to communicate their design process.

Analysis
Our research team received all Fall 2016 design journals from the individual instructors of each section of the Design Thinking course after the semester concluded.Of these, 92 design journals came with students' permission for evaluation.The full sample of 92 design journals included 44 journals from the integrated sections and 48 from non-integrated sections.We made note of the Design Thinking students' demographics at this stage to ensure a baseline similarity between both integrated and non-integrated groups: the population of students in integrated sections included 93 freshmen, 4 sophomores, and 1 junior; non-integrated sections comprised 45 freshmen, 12 sophomores, and 1 senior.
All collected artifacts were evaluated with the Engineering Design Process Portfolio Scoring Rubric, or EDPPSR (2011).To minimize researcher bias, all design journals were de-identified prior to evaluation, and researchers were blind as to which journals came from which sections.Grades for each assignment were not attached, which ensured there would be no grade-related biases in researchers' evaluations.All data identifying individuals and instructors were also removed.
Before scoring the full sample, two researchers independently evaluated approximately 22% of the journals using the EDPPSR and then analyzed their level of agreement on each element (Coots et al., 2017).Both raters had formerly taught multiple sections of the Design Thinking course, and were graduate students with interests in teaching design.This experience gave them the background needed to build appropriate expectations leading into the rating process.The inter-rater reliability of their independent scoring on this smaller sample, as determined via Cronbach's alpha values, was at least .75 for each rubric category and was .97 for the total score-an acceptable reliability coefficient (Nunnaly, 1978).After establishing an acceptable reliability coefficient, the raters split the remaining journals and each evaluated approximately one-half of them.
Once the full sample of 92 design journals had been scored, analysis was conducted on all rubric elements as well as on the overall summed scores.Based on descriptive statistics (Table 2) and visual inspection of the distributions, we considered the distribution of scores on each rubric element and the total score approximately normal.While this judgment satisfies the statistical assumptions for parametric statistics, the limited outcomes on each rubric element led us to apply non-parametric statistical tests which are more appropriate for nominal data (Aron, Aron, & Coups, 2009;MacFarland & Yates, 2016).Differences between the integrated and non-integrated course on each rubric element were tested using the Mann-Whitney U test.The total score was calculated as a sum of each element and had greater variation, while still being approximately normal.Therefore, we conducted an independent means t-test to consider a difference between the two-course types on overall design journal score.Ultimately, there was not a significant difference between integrated sections and non-integrated sections on any of the EDPPRS elements, or overall (Table 2).number expected).This follow-up research revealed that student design journals were consistently incomplete.Average completeness for this sample was only 59%.This average held true for both non-integrated and integrated sections.
From this follow-up investigation, we also learned that many teams arranged their design journals out of the expected chronological order.Rather than following the indicated template and building their design journals as a group as they worked through the project over eight full weeks, it appeared as though students assembled their team journals after the fact, filling in the blanks they could without concern for following chronological order.Rather than collaborating and sharing their individual projects during the term, students seem to have more often copied and contributed their portions of the design work individually at the end of the term.None of the design journals analyzed for completeness contained every assigned element.
Students may not have prioritized this design journal assignment for many reasons.The assignment itself may have been difficult for some to understand fully, or the assignment may have seemed minor in comparison with the larger final design project and presentation.It is also possible that aspects of the EDPPSR are not congruent with the assessment from the course.However, we did not use elements imperceptible in the design journals, and we believe the rubric elements do demonstrate good practices for documentation regardless of external assessment.Whatever the case may have been for students in these sections of Design Thinking, the assignments collected for this study do not reflect well-documented or satisfactorily complete design journals.

Conclusions and Directions for Future Research
Our initial research question involved asking whether or not students in the integrated version of a Design Thinking course learned to more effectively document and communicate their design process.Answering this question would determine whether or not the integrated course helped teach design thinking more effectively.The study described above shows that the integration appears to have made little difference to students' abilities to document their design process.
However, our project has also brought up serious concerns about the validity of using this set of largely incomplete design journals to measure students' abilities to document their design skills and demonstrate clear organized design thinking.That there were no differences for students in the integrated sections, and that the proportion of incomplete design journals was even across both section types do suggest that there is important work to be done in developing design documentation pedagogy.Incomplete entries in student design journals are missing data in much the same way that omitted survey questions might be problematic.As Tabachnick and Fidell (2007) noted with regard to missing data, "its seriousness depends on the pattern of missing data, how much is missing, and why it is missing" (p.62).Therefore, our follow-up as researchers and teachers of Design Thinking in Technology should be to more fully understand what student and/or teacher characteristics might predict these shortcomings in design documentation.In so doing we may identify aspects of the course or instruction that need to be improved broadly.On the other hand, we may identify exemplary strategies for design reflection, documentation, and communication.
As part of our continued efforts to develop the Integrated First-Year Experience, we are considering potentially worthwhile changes in how the design journal assignment is implemented and taught.We may also expand our study to include Design Thinking instructors' experience with and perspectives on the design journal assignment.Instructors with experience teaching and grading this assignment may have suggestions for better ways of encouraging students to complete the design journals thoroughly.It may help some students if the Design Thinking course fully standardized all requirements of the design journals, in order to make the end product easier to envision.While such a fixed structure may take away from the "design" or creative element that students are asked to engage with, offering footholds and scaffolding for these first-year students will hopefully guide students as they develop a stronger design thinking mindset.
We plan to replicate this study in coming semesters, drawing on a larger sample of potentially more complete artifacts.An analysis using artifacts collected from integrated and non-integrated Fall 2017 sections of Design Thinking is currently underway.This research and the teaching practices of Purdue's Polytechnic Institute, as well as those of other programs teaching principles of design in user-centered, project-based technology courses, would benefit greatly from further discussion on this important topic.

Table 1
Comparison of Non-integrated and Integrated Design Thinking Course Sections

Table 2
Average design journal scores for non-integrated and integrated sections, per rubric element.