Mapping class learning outcomes of the core curriculum to university learning goals at Michigan State University's School of Packaging

Class learning outcomes (CLOs) supporting the interdisciplinary curriculum of the School of Packaging (SoP) at Michigan State University (MSU) were mapped to competency‐based, programmatic learning outcomes (CPLOs) undergirded by broad learning goals of the University (MSU‐LGs). Six CPLOs related to the core curriculum, including the ability to design and evaluate packaging systems, as well as students' professionalism and teamwork skills, were developed using consensus building with all teaching faculty. Relationships, from specific to broad (i.e., CLOs to CPLOs to MSU‐LGs), were mapped with the SoP faculty. This mapping scheme (class‐specific CLOs supporting broader program CPLOs and, ultimately, MSU‐LGs) was developed to guarantee alignment of expectations for learning from the course to the packaging program to the MSU‐LGs. From Fall 2018 until Fall 2019, assessment tools, including rubrics and assignments intended to evaluate learning, were developed to assess core and elective courses offered by the SoP. Data collection of each student's performance was conducted utilizing Watermark's VIA software from Fall 2018 until Fall 2021 and continuous. Assessment of student performance provided evidence of learning across the SoP curriculum as well as how CLOs, delivered and assessed at the individual student level, translate into competence achieved at the programmatic and university levels.

tegic plan that, among other things, dictated the deployment of efforts and resources to improve the overall BS degree. Curriculum and learning assessment methods such as curriculum map with tabular matrices and quantitative assessments of student capabilities have been extensively used to evaluate students' learning (Herrmann & Leggett 6 ; Spencer et al. 7 ). In the process of creating a curriculum map, it is necessary to have outcome-based curricula and clear learning outcomes to facilitate the assessment at the end of the semester. McNeir and Spady define outcome-based curricula as the creation of class learning outcomes that focus on the exit skills expected of students at the end of each semester. Spady 8 also elucidates that an outcome-based educational system focuses on optimizing curriculum by collecting and analyzing student learning outcomes and feedback. Outcome-based approaches to curricular framing and learning assessment have been utilized in education for decades (Harden 9 ; Spady 8 ) and employed by various disciplines to drive accreditation and student learning (Harden 10 ; Uchiyama & Radin 11 ; Wang 12 ). The approach, kindred to that used in a backward design process, emphasizes the exit outcomes needed by students rather than the objectives when designing a curriculum (Harden 9,13 ; McNeir 14 ; Spady 8 ). The SoP has a long history of a strong relationship with external stakeholders; as such, we have historically framed student assessment and success in outcome-based curricula.
Under this context, learning outcomes are defined as the knowledge, skills, and attitudes/characteristics the students should acquire by the end of a particular assignment, class, course, or program (Centre for Teaching Support & Innovation University of Toronto 15 ; Hoque 16 ).
F I G U R E 1 Cumulative graduates by year graduating with Bachelor of Science (BS), Master of Science (MS), and Doctor of Philosophy (PhD) degrees from the School of Packaging (SoP) at Michigan State University as April 2022. The inset shows cumulative graduates of the MS and PhD degrees, excluding the BS degree.
As part of the mentioned planning efforts, a full evaluation of the core packaging curriculum was conducted to identify curriculum gaps and other shortcomings. Evidence assessed included exit surveys of students and feedback from the external packaging industry advisory board and the alumni association, which were used to (a) develop the competency-based, programmatic learning outcomes (CPLOs), which are the outcomes that students should master by the end of their pack-   19 ), but they can also be shown in graphical formats (Seering et al. 20 ; Spencer et al. 7 ). The tabular and graphical formats were implemented on the SoP curriculum map. The presented discussion and analysis exclude courses that are classified as packaging electives and those that are not taught by packaging faculty since several of these courses fall outside the SoP jurisdiction, and we lack control of their deliverable. In an ideal situation, the entire MSU packaging curriculum (i.e., the 120 credits and not only the credits under SoP control, but c. 52 credits) could be mapped. However, at this point, this is not possible. This paper presents the methodology that we used to conduct the curriculum mapping including correlation matrix and chord diagram and reports the results from the curriculum assessment of the core curriculum between Fall 2018 and Fall 2021. It also provides information on the curriculum arrangement and teaching system in the field of packaging programs and how faculty can improve the curriculum and quantify the learning outcomes on this basis.

| Learning outcome mappings
As part of the curriculum assessment plan, six CPLOs that had a consensus agreement regarding their importance as competencies for F I G U R E 2 The current School of Packaging curriculum including core and elective courses F I G U R E 3 Different levels of learning outcomes for the packaging curriculum mapping at the School of Packaging, Michigan State University exiting students were created as shown in Table 1. CPLO-1 to CPLO-6 were determined to be required for all packaging students and, as such, assigned to the packaging core courses (listed in Figure 2). CPLO-1 to CPLO-6 were initially approved by the SoP teaching faculty. The following results and discussion section concentrate on the core curriculum and CPLO-1 to CPLO-6 (excluding the elective courses). Nonetheless, the mapping and evaluation process for the electives followed the same process as those explained herein.
Instructors initially developed CLOs for packaging courses for which they had the primary responsibility to teach; drafted CLOs were discussed, revised, and approved by all the instructors through consensus discussion and in voting faculty meetings. CPLOs and CLOs for the core curriculum were approved in a voting faculty meeting in the  Table 2, which provides an excerpt of a single course's CLOs (PKG 101). Graphical data visualization in the form of a hierarchical circular layout (Holten 22 ) (hereafter referred to as a chord diagram) of the CLO relationships was created based on a modified schemaball code (Komarov 23 ) using MATLAB program R2021b (MathWorks, Natick, MA, USA). Relationships between the CPLOs and the MSU-LGs (listed in Table 3), and the CLOs and CPLOs were initially established in table formats and later transformed into a graphical format. By matching CLOs, CPLOs and MSU-LGs to establish links between them, helps us refine the macro (University level) goals for student development into micro targets related to packaging courses. It can also be considered that students improve other skills such as their communication skills and professionalism by completing certain courses in packaging programs to achieve a small task (such as a group project), to achieve CPLOs (e.g., CPLO-5: Communicate effectively considering diverse audiences) and MSU-LG (Effective Communication) related to general communication competency.

| Learning outcome assessments
Rubrics and questionnaires to evaluate learning outcomes affiliated with the core courses were created and implemented using a learning outcomes measurement software (VIA by Watermark, Watermark, T A B L E 1 School of Packaging Bachelor of Science degree competency-based, programmatic learning outcomes (CPLOs)

CPLO-1: Evaluate packaging systems
Evaluate and choose materials, packaging systems, and components by applying and interpreting scientific tests, writing and interpreting specifications, and using and recommending appropriate test protocols.

CPLO-2: Analyze trade-offs in packaging systems
Analyze and prioritize product, packaging line, distribution system, environmental footprint, marketing, financial implications, user needs, and additional trade-offs to create and/or provide innovative, efficient, sustainable, legally compliant, and cost-effective packaging systems.

CPLO-3: Design innovative and sustainable packaging systems
Design innovative and sustainable solutions to package and packaging systems-related problems and future needs.

CPLO-4: Manage projects in diverse teams
Formulate and manage projects with the ability to work in diverse teams to achieve common and successful outcomes.

CPLO-5: Communicate effectively considering diverse audiences
Communicate effectively considering diverse audiences in a variety of situations with a variety of media.

CPLO-6: Conduct oneself in a professional and ethical manner
Conduct oneself in a professional and ethical manner, exhibiting values such as civility and respect for others, honesty, integrity, accountability, maintenance of confidentiality, etc. and allows administrators to assess the student learning outcomes, also connecting with CPLO and MSU-LG. Rubric development and CLO assessment were created by the faculty responsible for teaching the class with help of a core team working on the curriculum mapping.
Instructors created rubrics for specific instrument or assignment based on the elements or tasks that students are expected to complete-see example on Table S2 for PKG 101 and Table S3  learning of trade-offs analysis and the learning outcomes, which provided a direction for subsequent curriculum improvement (CLOs and CPLOs assessments for these courses are shown in Figures S1-S4).
A schemaball type web (Chord diagram), as further discussed in the result section, was used to show the correlation between each core course. Node colors were used to indicate the degree of relevance between core courses (the more connections, the darker the color). More correlation means that students need additional knowledge from other courses when they take this course, or this course is the basis for future learning on other courses. So, instruments for assessment of these CLOs were designed to collect evidence of learning and mapped the achievement on these CLOs. A detailed explanation is mentioned in Section 3. Analytical thinking AT1: Acquires, analyzes, and evaluates information from multiple sources.

| Learning outcome mappings
AT2: Synthesizes and applies information within and across disciplines.
AT3: Identifies and applies, as appropriate, quantitative methods for defining and responding to problems.
AT4: Identifies the credibility, use, and misuse of scientific, humanistic, and artistic methods.
Cultural understanding CU1: Reflects on experiences with diversity to demonstrate knowledge and sensitivity.
CU2: Demonstrates an awareness of how diversity emerges within and across cultures. Relationships between the University level goals (MSU-LGs) and the SoP CPLOs were also established through consensus; these relationships are depicted in Figure 6A, and a matrix that mapped all connections (i.e., CLOs to CPLOs to MSU-LGs) was created. An excerpt of the matrix is shown in Figure 6B. The full graphical relationship is shown in Figure 7, where the thicker lines between CPLOs and packaging classes indicate a higher number of linkages. This data visualization helps determine if CPLOs or MSU-LGs are linked to some courses (through the CLOs) more than other courses, and the curriculum can be adjusted for a better balance following CLOs all the way to MSU-LGs.
This exercise yields a useful visualization to examine how resources are being allocated across an entire curriculum. In our case, CPLO-1 (evaluate packaging systems) is garnering significantly more effort than CPLO-4 (manage projects in diverse teams). It is not necessarily the case that each and every programmatic outcome require the same level of time and energy but this yields a useful tool for discussion and, when necessary, revision. From a wider perspective, our map also suggests that University goal EC1 is exercised with less frequency in our classrooms than other University level goals. All of this F I G U R E 4 Pre-requisite and correlation between CLOs in the core packaging curriculum. Pre-requisite indicates mandatory mapping between courses; dashed line indicates relations between CLOs in a previous and forthcoming course.
provides a starting point for discussion for a complex system of connected topics and continuous growth mindset.

| Learning outcome assessments
The implementation and mapping of learning outcomes are not only useful at the programmatic level (curriculum development and refinement) but are also useful for more granular evaluations. For specific courses, the learning outcomes are reflected by the accomplishment of lab, assignments, group discussion, or final projects. The quantification of learning outcomes can be used to assess student growth and instructor efficacy and encourage institutions to propose improvements. Students achieve the corresponding learning goals by completing the course tasks, and the teaching quality of the course can be evaluated by counting the completion of the students' goals.  Table S1). We can observe that in Fall 2018 of the 384 times that CLO-7 was assessed 62 times the students did not meet the requirements. This was improved through consecutive years, but this outcome seems to provide students more difficulty than others assessed; observations like this can provide the instructor with insight that may otherwise go unnoticed during the period of a busy semester than a total score of a project for final evaluation (the competency may be embedded within an assessment that handles more than one competency and, as such, have the signal lost).
It is also important to indicate that CLO-7-create and prototype packaging systems-was not evaluated in SS20 due to beginning of the COVID-19 pandemic and the lack of time to switch to virtual creation of prototypes. During FS20 and SS21, CLO-7 was modified, and prototypes were created virtually. By FS21, the CLO was required to be conducted in person. In the same manner that we can assess the The results were normalized as percentage as shown in the x axis of the figures. The CPLO assessment in Figure 9 shows that more than 95% of the students met all the CPLOs in Fall 2019. The "met" criteria were allocated to 70% passing of the outcomes as established by the SoP instructors. The MSU-LG assessment in Figure 10 shows that all MSU-LGs except the EC2 achieved a 90% met status in Fall 2019.
The data indicate the importance of mapping across the curriculum and not only in selected courses since a robust measurement system allows for determining learning gaps in early courses (the SoP Bachelor of Science degree curriculum has been included in Supporting Information S13-S15).
Although CLO, CPLO, and MSU-LG data were collected during COVID, some selective CLOs which require experimental work and lab could not be assessed (e.g., PKG 323 Packaging with Plastics  Accreditation outcomes, learning outcomes, modules, and other entities were used for data analysis in different mapping models. The network models achieved visualization and provided help for the development of curriculum design, student planning, advising, and quantitation of teaching outcomes. In our case, the mapping between correlated CLO across CPLO was done and assisted by faculty, which guided the selection of this correlation. After the initial selection, they were connected through a schemaball type web (see Figure 5), and the main correlated points were used for developing assessment instruments that could be used for data collection as shown in This correlates with an introduction, understanding, practice, and creation structure of learning. This is also similar to the concept of the allocation of courses in a training itinerary discussed in Simon de Blas et al. 25 's study related to the complex network analysis tool for curricula design.
In our mapping and collection of data, the VIA system was instrumental in data collection. To facilitate the collection of CLOs from many instructors, it is crucial that a class learning management system (CLMS) is in place so that rubrics and assessments mapped to standards can be created, delivered, and assessed. Instructors can guide students to submit coursework or lab reports to the VIA system and can also evaluate the course completion of each student in each semester in the system, and these data can reflect the completion of CLOs, CPLOs, and MSU-LG. These data can then be aggregated and processed to form the reports shown in Figures 8-10.
Taking PKG 485-Packaging Development as an example, as shown in Figure 8, CLO-6 "Use a variety of media to communicate effectively with diverse audiences," except for both SS19 and FS21, almost all the students achieved their study goals, and it can be seen from Figure 7, CLO-6 is related to CPLO-5 "Communicate effectively considering diverse audiences." Through the mapping shown in Figure 6, we can find that CPLO-5 is related to CU1, CU2, ECO1, ECO2, IR1, and IR2 of the MSU-LG. This means that if a student acquires and it is competent with the skills of CLO-6 required in PKG 485, then this student has completed the mapping of CPLO-5, thus also enabling achievement of the university-level learning goals, which means through the study of this course (such as completion of projects, group work, and presentations), the student has acquired the ability to communicate effectively in the context of different audiences.
We should remember that there are several CLOs for many courses that were previously assessing the student progress during the packaging program. The mapping from CLO to CPLO and MSU- LG provides an additional subjective perspective of the opinion and thinking of the faculty in the program and the common knowledge in the field to map how these CLOs are responsible for the final delivery of CPLOs and MSU-LGs. A mapping methodology can help move away from "silo learning" course to curriculum competency learning outcomes. However, how to clarify the complex relationship between courses and quantify the impact of courses on students' achievement of their study/career goals and personal development remains to be explored and in need of extensive research.

| FINAL REMARKS
The curriculum assessment for any academic program is a journey and requires continuous evaluation and adjustment so that it keeps relevant and up to date. We have shown the SoP's initial steps on this journey and a methodology for assessing and mapping CLOs to CPLOs to MSU-LGs. Collection of data in this fashion offers the promise of evaluating learning at several levels. It can potentially be used: (a) at the student level to assess learning and growth over time; (b) at the classroom level to assess how pedagogical changes (over time) impact student learning; and (c) at the program level to assess how well the curriculum is meeting stakeholder demands and disciplinary requirements for student learning. Ultimately, this type of data can be used to assess and improve the curriculum and for degree competency certification. Although we have not yet measured the effectiveness of changes on the student experience, the process of identifying outcomes and mapping the curriculum has been fruitful in revealing gaps that we can effectively bridge and start working.