A Framework for Efficient and Accurate Automated CLO and PLO Assessment

Accurate and efficient learning outcome assessment is crucial for ensuring high-quality education, but traditional methods can be time-consuming, error-prone, and inconsistent. We developed a novel Excel Macro-enabled framework for automating the evaluation of Course Learning Outcomes (CLOs) and Program Learning Outcomes (PLOs) in higher education. The framework consists of two Excel Macro-enabled workbooks. The course section workbook guides instructors through the assessment process, automatically calculates CLO achievement levels


INTRODUCTION
Ensuring that students acquire the intended knowledge, skills, and values is paramount in the dynamic world of higher education.This necessitates effective assessment of Course Learning Outcomes (CLOs) and Program Learning Outcomes (PLOs), which capture the intended knowledge, skills, and values students should acquire by the end of a course or program, respectively.However, traditional assessment methods often fall short, proving to be time-consuming, prone to errors, and lacking consistency in application across different courses or programs [1,2], leading to inaccurate evaluations, hinder program improvement efforts, and ultimately impede student success.A critical need arises for innovative solutions that streamline and enhance learning outcome assessment, in [3][4][5].This paper introduces a novel automation framework designed to address these challenges.Built on readily accessible Excel Macro-enabled workbooks, the framework simplifies and automates many of the assessment tasks within the direct assessment methods, empowering both instructors and coordinators to:  Reduce the time and effort invested in assessment: Manual calculations and report generation are replaced with automated processes, freeing up valuable time for educators to focus on teaching and student interaction [6].
 Minimize errors and inconsistencies: Standardized calculations and automated data aggregation ensure accuracy and consistency across assessments, providing reliable data for program evaluation and improvement.
 Facilitate data-driven decision making: Comprehensive reports generated by the framework provide rich insights into CLO and PLO achievement levels, enabling informed decisions about curriculum development, instructional strategies, and resource allocation.
The framework's efficacy is demonstrated through its successful implementation in the Faculty of Computer and Information Technology (FCIT) at Northern Border University (NBU).Used by three programs and lauded by accreditation agencies like NCAAA [7] and ABET in [8], it stands as a testament to the framework's potential to revolutionize learning outcome assessment in higher education institutions.This paper delves deeper into the framework's design, implementation, and impact.We explore the functionalities of the two Excel workbooks, one for course sections and another for courselevel assessment.We showcase how the framework automates calculations, generates reports, and aggregates data from multiple sections to provide comprehensive insights into program-level PLOs achievement.Furthermore, we present a detailed case study of the framework's implementation in the FCIT, highlighting its demonstrable effects on reducing assessment time, minimizing errors, and facilitating data-driven program improvement.Finally, we discuss the broader implications of the framework for higher education institutions seeking to streamline and enhance their CLO and PLO assessment practices, particularly those adhering to similar national qualification frameworks.This framework lays the way for a future in which learning outcome assessment is not a burden, but a strong instrument for driving continuous improvements in student learning and program quality.

II. CLO AND PLO ASSESSMENT AUTOMATION FRAMEWORK
Ensuring students achieve high-quality learning outcomes is paramount in today's rapidly evolving higher education landscape, where inadequate learning can hinder career prospects and personal growth [9,10].Central to this goal is the effective assessment of CLOs and PLOs.However, traditional assessment methods often fall short, proving to be time-consuming, prone to errors, and lacking consistency.This hinders accurate evaluation, limits data-driven decision making, and poses challenges for accreditation requirements.To address these challenges, we present the novel CLO and PLO Assessment Automation Framework.This framework leverages readily available Excel Macro-enabled workbooks to streamline and automate many of the assessment tasks, empowering both instructors and coordinators to:  Reduce the time and effort invested in assessment, freeing up valuable resources for teaching and student interaction.
 Minimize errors and inconsistencies through standardized calculations and data aggregation, ensuring reliable data for program evaluation and improvement.
 Facilitate data-driven decision making by providing rich insights into CLO and PLO achievement levels, enabling informed decisions about curriculum development, instructional strategies, and resource allocation.
 Simplify accreditation compliance by generating reports tailored to accreditation agency requirements, minimizing data collection and reporting burdens.
This framework operates through two key components: Course Section Assessment Workbook and Course Assessment Automation Workbook.Course section assessment workbook home page.

A. Course Section Assessment Workbook
This workbook guides instructors through the assessment process for each individual course section.It automates calculations, generates reports, and facilitates evidence upload, empowering instructors to focus on delivering quality instruction and providing insightful feedback to students.This instructor-facing workbook simplifies and automates the assessment of CLOs within individual sections.Key features include:  Automated Calculations: Manual calculations are implemented through pre-defined weighting schemes and rubrics.
 Individual Student Reports: Detailed reports are generated for each student, summarizing performance and providing feedback.
 Cloud-based Evidence Upload: Securely upload evidence of assessments via integrated cloud links.
 Marks Summary for HoD Approval: Streamline mark submission with a dedicated report for Head of Department (HoD) approval.
The Course Section Assessment Workbook includes hidden (protected to be visible by the instructor) sheets and visible sheets can be displayed or accessed using links from the home page as shown in Figure 1.Hidden sheets act like a database used to store the registration data, the study plan, and the CLO-PLO alignment: Accessible Sheets allowing section instructor to specify the assessment plan of the course, upload marks of each student in each assessment tools relative to CLOs and upload evidence relative to each assessment tools, and obtain CLOs and PLOs, SO attainment and the marks summary to be shared with head of the department:  Section information (main application interface): Serves as the primary interface for instructors, providing access to other sheets and functionalities within the workbook.
 Student list (automatically generated): Displays the list of students enrolled in the selected section, extracted from the hidden "Student registrations" sheet, ensuring accurate student information.
 CLO-PLO alignment (automatically generated): Presents the alignment of CLOs with PLOs for the specific course, providing a clear overview of learning objectives and their contribution to program outcomes.

 Assessment Plan (assessment tool coordination table):
Facilitates coordination among instructors by aligning assessment tools and weighting schemes across different course sections, promoting consistency in assessment practices.
 Assessment Tools sheets (Quizzes, Assignments, Exams, etc.) with cloud links to gather evidence: Provide dedicated spaces for instructors to enter assessment scores for various assessment methods, along with cloud links for secure upload of assessment evidence, streamlining evidence collection and storage. Marks Summary (for HoD approval): Presents a comprehensive summary of student marks for review and approval by the Head of Department (HoD), streamlining mark submission and approval processes. Centralized data management: Serves as a repository for course information, alignment matrices, and assessment data, facilitating program-level analysis and decisionmaking.
 Compliance support: Generates XLSX reports (without macros) for seamless sharing with accreditation agencies, streamlining compliance efforts.Course-level CLO and PLO assessment automation is achieved through a custom VBA script function named GenCLOsPLOsAssCourse() (Figure 7).This function simplifies data aggregation by prompting the user to select multiple section-level assessment workbooks.It then iterates through each workbook, extracting relevant data such as section ID, student count, grade distributions, CLO and PLO mapping, and CLOs with assigned marks.The extracted data are then populated into corresponding worksheets within a single course-level assessment workbook.Notably, the function dynamically handles branches (male/female) and department information, utilizing it to select the appropriate worksheet for CLO-PLO mapping and tracking student and section counts accordingly.Overall, this function streamlines the generation of comprehensive course-level assessments by consolidating data from multiple sections.
The framework seamlessly integrates instructor and coordinator tasks, streamlining assessment processes from the individual section level to program-wide analysis.Instructors leverage the Course Section Assessment Workbook to manage student information, enter scores, visualize CLO and PLO attainment, upload evidence, and generate reports for approval.Coordinators, in turn, utilize the Course Assessment Automation Workbook to effortlessly import section files, visualize course-level statistics and grade distributions, analyze CLO and PLO achievement across sections, and generate comprehensive reports for quality assurance purposes.

III. IMPLEMENTATION AND CASE STUDY The implementation of the CLO and PLO Assessment
Automation Framework at FCIT necessitated meticulous integration with the current procedures.The initial phase entailed engaging in collaboration with teachers and coordinators to comprehend their assessment methodologies and requirements.As a result, the Assessment Plan sheet was modified to incorporate the use of current assessment tools and weighting systems, ensuring a seamless transition without causing any disruption to established procedures.
Challenges were inevitable.The initial concerns encompassed a range of technical anxieties and apprehensions over the division of effort.To tackle these issues, extensive training sessions were carried out, emphasizing the intuitive interface and the potential for significant time savings.In addition, open communication channels were established to handle technical issues and give continuous assistance.This was necessary because the framework is built on the NBU licensed Office 365, and some instructors are reluctant to upgrade to the required one.The framework's effectiveness was validated through its application in three diverse programs: Information Systems (IS), Computer Sciences (CS), and Information Technology (IT).Here's a glimpse into the transformative impact: The achievements observed in each of these programs provide strong evidence of the potential of the framework.In addition to enhancing efficiency and precision, it facilitates data-driven decision making, which fosters ongoing program enhancement and guarantees that graduates possess the necessary competencies and understanding to prosper in the contemporary society.

IV. COMPARISON WITH EXISTING WORKS
To demonstrate the distinctiveness and value of our framework, we present a comparison with existing solutions for CLO and PLO assessment.This comparison highlights the unique features and advantages our framework offers, positioning it as a valuable tool for higher education institutions.While existing commercial learning management systems like Blackboard Learn [11] and Desire2Learn [12] offer assessment tools and reporting features, they lack the tailored focus on CLO and PLO alignment and reporting found in our framework.Open-source solutions like ELMS LN [13] and OAT [14], while flexible, require technical expertise and don't directly address CLOs and PLOs.Research like Al-Naki et al.'s [12] explores CLO-PLO mapping automation using NLP, but lacks broader assessment features.Others [4,6], propose models and frameworks for program improvement through assessment data, but lack the specific software implementation offered by our Excel-based solution.Our framework's strengths lie in its ease of use, specific focus on CLO and PLO, data-driven decision-making capabilities, and customizability, making it a unique and valuable tool for higher education institutions.
The comparison presented in Table I underscores the unique advantages of our CLO and PLO assessment automation framework.Its specific focus on CLOs and PLOs, user-friendly Excel-based interface, comprehensive assessment automation features, data-driven decision-making capabilities, and customizable design make it a compelling choice for higher education institutions seeking to streamline and enhance their assessment processes.By effectively addressing the complexities of CLO and PLO assessment, our framework empowers institutions to make data-informed decisions that drive continuous program improvement and ensure student success.

TABLE I. COMPARISON TABLE
The proposed CLO and PLO Assessment Automation Framework demonstrates the significant influence that technology may have on the fields of education.The advantages of the proposed framework are indisputable: instructors and coordinators experience a decrease in burden, assessment accuracy and consistency are enhanced, and above all, there is a notable transition towards curriculum development guided by data and student learning.These benefits are especially significant in academic institutions, which prioritize the correlation between program outcomes and student achievement.The framework's potential transcends the confines of FCIT.Its adaptability and modular design make it a compelling solution for other higher education institutions, as evidenced by its recent adoption by the Electrical Engineering (EE) program at NBU for term 2, AY 2023/2024.Disseminating this framework, with appropriate customization and contextspecific adjustments, can foster a broader culture of assessment excellence.This is particularly relevant within institutions like NBU, which emphasize program outcome-student achievement alignment.However, the framework's flexibility extends beyond specific frameworks, making it a valuable tool for fostering a more comprehensive approach to assessment and program improvement in diverse educational contexts.
While the framework demonstrates significant strengths in streamlining CLO and PLO assessment, it also presents some limitations that require ongoing attention.To ensure the framework's correctness and effectiveness, it is necessary to do periodical modifications at the start of each term.The QAAC faces a substantial burden since they are required to manually collect and update data from Banner, including information on available sections and registrations.Additionally, they must align CLOs and PLOs and prepare data for the workbooks.The manual approach is both time-consuming and susceptible to errors.To mitigate this load and guarantee seamless adoption in the future, other options could be considered:  Creating automatic data import tools: By optimizing the process of retrieving data from Banner, we can greatly reduce the need for manual labor and mitigate the potential for errors.Collaboration with IT teams is necessary to develop interfaces that efficiently send pertinent data to the workbooks.
 Integrating pre-filled templates: Offer pre-configured templates for aligning CLOs and PLOs, minimizing the requirement for human data input and conserving significant time.
 Discovering the capabilities of cloud-based data storage: Transferring data storage to the cloud could streamline access and communication among instructors and coordinators, obviating the necessity for manual file transfers and guaranteeing data availability across departments.
Through the implementation of these solutions, their attention can be directed towards the analysis of data and the provision of vital insights to enhance student learning and the efficacy of the program.This will guarantee the ongoing success of the framework and decrease the administrative workload linked to quarterly upgrades.
Essentially, the CLO and PLO Assessment Automation Framework serves as a catalyst for a fundamental change in our assessment strategy and enhances educational quality, going beyond being only a technological solution.By acknowledging its capacities and exploring further possibilities for expansion, we may collectively cultivate an educational setting in which data not only improves the assessment process, but also the entire procedure of nurturing confident and successful graduates.


Program study plan: Provides a comprehensive overview of all courses within the program, including details such as course names, credit hours, prerequisites, and levels. Program PLO-SO alignment: Maps the program's PLOs to external frameworks such as ABET's Student Outcomes (SOs), ensuring alignment with accreditation requirements and industry standards. Course-PLO alignment: Specifies the alignment of each course with specific PLOs, indicating the degree to which each course contributes to the achievement of programlevel outcomes. CLO-PLO Alignment: Delineates the alignment between individual CLOs and the corresponding PLOs, ensuring coherence between course-level and program-level learning objectives.Course Sections (updated each term): Maintains a list of course sections offered in the current term, including section IDs, instructors, and timetables, ensuring up-to-date information for assessment. Student Registrations (updated each term): Records student enrollment in each course section, providing accurate student lists for assessment activities. Coordinators (updated each term): Stores contact information for course coordinators, facilitating communication and collaboration in assessment processes. Process Sheet (parameters, data, calculations): Contains various parameters, filters, and calculations used within the workbook, enabling automated processes and data manipulation.

Fig. 4 .
Fig. 4.Course section marks summary sheet.CLO Attainment (automatically generated): Summarizes the attainment levels of CLOs for each student, calculated based on entered scores and pre-defined weighting

Fig. 5 .o
Fig. 5.Uploading assessment data for an assessment tool sheet.B.Course Assessment Automation WorkbookThis workbook acts as a central hub for coordinators, aggregating data from all course sections and providing comprehensive insights into program-level PLO achievement.It offers automated data analysis, report generation, and facilitates data sharing with stakeholders, simplifying program evaluation and accreditation processes.This workbook aggregates data from all sections to provide comprehensive program-level insights relative to the course:  Effortless Data Aggregation: Seamlessly imports data from individual section files, minimizing manual entry and errors.Rich Reports and Visualizations: Generates reports and charts showcasing CLO and PLO achievement across sections and student cohorts.Accreditation Compliance: Simplifies data collection and reporting for accreditation agencies.Centralized Data Management: Maintains an organized repository of course information, alignment matrices, and assessment data for future reference.The Course Assessment Automation Workbook has the following key features:  Data Aggregation: Seamlessly imports data from multiple section files, eliminating manual entry and ensuring consistency across sections.Automated Report Generation: Automatically generates informative reports, including:

o
Reduced Assessment Time: o Instructors from 3 departments reported a 50% reduction in time spent on calculations and report generation.All course coordinators noted a 30% decrease in workload associated with data aggregation and analysis.Enhanced Accuracy: o Automated calculations eliminated human error, leading to greater confidence in data integrity.o Standardized formulas across sections ensured consistency in attainment calculations. Data-Driven Decision Making: o PLO-SO attainment insights in IT helped identify areas for curriculum revision, aligning program outcomes with industry expectations.o Real-time CLO attainment data in IS facilitated targeted interventions for struggling students, boosting their performance.o CS program coordinators utilized grade distribution reports to identify sections requiring additional resources or faculty development.