Skip to main content
SpringerLink
Log in

Automating the mapping of course learning outcomes to program learning outcomes using natural language processing for accurate educational program evaluation

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

Quality control and assurance plays a fundamental role within higher education contexts. One means by which quality control can be performed is by mapping the course learning outcomes (CLOs) to the program learning outcomes (PLO). This paper describes a system by which this mapping process can be automated and validated. The proposed AI-based system automates the mapping process through the use of natural language processing. The framework underwent testing using two actual datasets from two educational programs, and the findings were promising. A testament to the potential of the suggested framework was the precision of the mapping detected (83.1% and 88.1% for the two programs, respectively) compared to the mapping performed by the domain experts. A web-based tool was created to help teachers and administrators execute automatic mappings (https://dsaluaeu.github.io/mapper.html). The data and software used in this research project can be found at the following URL: https://github.com/nzaki02/CLO-PLO.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

The datasets used during the current study are available in the GitHub repository, [https://github.com/nzaki02/CLO-PLO].

Abbreviations

CLO :

Course learning outcome

PLO :

Program learning outcome

NLP :

Natural language processing

AI :

Artificial Intelligence

QA :

Quality assurance

HEI :

Higher education institutional

ILO :

Institutional learning outcomes

ABET :

Accreditation Board for Engineering and Technology

BERT :

Bidirectional Encoder Representations from Transformers

References

  • Alshanqiti, A., Tanweer, A., Mohamed, B., Abdallah, N., & Ahmad, T. (2020). A rule-based Approach toward automating the assessments of academic curriculum mapping. International Journal of Advanced Computer Science and Applications (IJACSA), 11(12). https://doi.org/10.14569/IJACSA.2020.0111285

  • Classification: Precision and Recall. (n.d.). Retrieved July 23, 2022, from https://developers.google.com/machine-learning/crash-course/classification/precision-and-recall

  • Devlin, J., Chang, M., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv. https://doi.org/10.48550/arXiv.1810.04805

  • Honnibal, M., Montani, I., Van Landeghem, S., & Boyd, A. (2020). SpaCy: Industrial-Strength Natural Language Processing in Python. Zenodo. Retrieved from https://spacy.io/. Accessed 10 July 2022

  • Ibrahim, W. A., Atif, Y., Shuaib, K., & Sampson, D. (2015). A web-based course assessment tool with direct mapping to student outcomes. Educational Technology & Society, 18(2), 46–59.

    Google Scholar 

  • Immerstein, R., Hasleberg, H., & Eri, G. (2020). Quality assurance for work placement in higher education. In IEEE Global Engineering Education Conference (EDUCON) (pp. 1096–1101). IEEE. https://doi.org/10.1109/EDUCON45650.2020.9125156

  • Lam, B. H., & Tsui, K. T. (2014). Curriculum mapping as deliberation – examining the alignment of subject learning outcomes and course curricula. Studies in Higher Education, 1371–1388. https://doi.org/10.1080/03075079.2014.968539

  • Levenshtein, V. I. (1966). Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10(8), 707–710.

    MathSciNet  Google Scholar 

  • Linden, J., Annala, J., & Coate, K. (2017). The role of curriculum theory in contemporary higher education research and practice. In J. Huisman & M. Tight (Eds.), Theory and Method in Higher Education Research (pp. 137–154). Emerald Publishing Limited. https://www.bibsonomy.org/bibtex/220546d80ce76f58c6ef6ece9dd5f5056/jimregan. Accessed 15 July 2022.

  • pandas.DataFrame — pandas 1.4.3 documentation. (n.d.). Retrieved July 6, from https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html

  • Plaza, C. M., Draugalis, J. R., Slack, M. K., Skrepnek, G. H., & Sauer, K. A. (2007). Curriculum mapping in program assessment and evaluation. American Journal of Pharmaceutical Education, 71(2). https://doi.org/10.5688/aj710220

  • scikit-learn Machine Learning in Python. (n.d.). Retrieved July 16, 2022, from https://scikit-learn.org/stable/

  • Shafi, A., Saeed, S., Bamarouf, Y. A., Iqbal, S. Z., Min-Allah, N., & Alqahtani, M. A. (2019). Student outcomes assessment methodology for ABET Accreditation: a case study of computer science and computer information systems programs. IEEE Access: Practical Innovations, Open Solutions, 7, 13653–13667. https://doi.org/10.1109/ACCESS.2019.2894066

    Article  Google Scholar 

  • Spencer, D., Riddle, M., & Knewstubb, B. (2012). Curriculum mapping to embed graduate capabilities. Higher Education Research and Development, 31(12), 217–231. https://doi.org/10.1080/07294360.2011.554387

    Article  Google Scholar 

  • Supraja, S., Hartman, K., Tatinati, S., & Khong, A. W. (2017). Toward the automatic labeling of course questions for ensuring their alignment with learning outcomes. In Proceedings of the 10th International Conference on Educational Data Mining. International Educational Data Mining Society. https://eric.ed.gov/?id=ED596615. Accessed 20 July 2022.

  • Treadwell, I., Ahlers, O., & Botha, G. (2019). Initiating curriculum mapping on the web-based, interactive learning opportunities, objectives and outcome platform (looop). African Journal of Health Professions Education, 11(1), 27–31. https://doi.org/10.7196/AJHPE.2019.v11i1.1073

    Article  Google Scholar 

  • Ujkani, B., Minkovska, D., & Stoyanova, L. (2021). Using natural language processing for quality assurance purposes in higher education. In IV International Conference on High Technology for Sustainable Development (HiTech) (pp. 01–04). IEEE. https://doi.org/10.1109/HiTech53072.2021.9614206

  • Vajjala, S., Majumder, B., Gupta, A., & Surana, H. (2020). Practical Natural Language Processing. O’Reilly Media, Inc.

  • Veltri, F., Webb, W., Matveev, Alexei, G., & Zapatero, G. (2012). Curriculum mapping as a tool for continuous improvement of IS curriculum. Journal of Information Systems Education, 22(1), 31–42.

    Google Scholar 

  • Watson, S., Steketee, C., Mansfield, K., Moore, M., Dalziel, B., Damodaran, A., Walker, B., Duvivier, R. J., & Hu, W. (2020). Curriculum mapping for health professions education: a typology. Focus on Health Professional Education, 21(1), 91–113. https://doi.org/10.11157/fohpe.v21i1.364

    Article  Google Scholar 

  • Winkler, W. E. (1990). String comparator metrics and enhanced decision rules in the fellegi-sunter model of record linkage. In Proceedings of the Section on Survey Research Methods (pp. 354–359). https://www.bibsonomy.org/bibtex/2a2dcfaedc01b2ffcab7afaffc6bd03bc/sam_chapman. Accessed 15 July 2022.

Download references

Funding

This work was done based on self-funding.

Author information

Authors and Affiliations

  1. Department of Computer Science and Software Engineering, College of Information Technology, United Arab Emirates University, 15551, Al Ain, United Arab Emirates

    Nazar Zaki & Sherzod Turaev

  2. Big Data Analytics Center, United Arab Emirates University, Al Ain, United Arab Emirates

    Nazar Zaki & Anusuya Krishnan

  3. Department of Info. Systems and Security, College of Information Technology, United Arab Emirates University, 15551, Al Ain, United Arab Emirates

    Khaled Shuaib

  4. Artificial Intelligence Research Center (AIRC), College of Engineering and Information Technology, Ajman University, Ajman, United Arab Emirates

    Elfadil Mohamed

Authors
  1. Nazar Zaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sherzod Turaev
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Khaled Shuaib
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anusuya Krishnan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elfadil Mohamed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nazar Zaki.

Ethics declarations

Conflict of interest

Not applicable.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zaki, N., Turaev, S., Shuaib, K. et al. Automating the mapping of course learning outcomes to program learning outcomes using natural language processing for accurate educational program evaluation. Educ Inf Technol 28, 16723–16742 (2023). https://doi.org/10.1007/s10639-023-11877-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-023-11877-4

Keywords

Search

Navigation