Skip to main content
SpringerLink
Log in

Diagnostic Evaluation of MOOCs Based on Learner Reviews: The Analytic Hierarchy Process (AHP) Approach

  • Conference paper
  • First Online:
Blended Learning: Educational Innovation for Personalized Learning (ICBL 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11546))

Included in the following conference series:

Abstract

The evaluation of MOOCs (massive open online courses) is needed to improve their design quality and to inform learners regarding course selection. In this paper, we proposed and validated an Analytic Hierarchy Process (AHP) Approach based on standardized rubric, expert feedback, data mining and emotion detection to systematically and diagnostically evaluate the quality of MOOCs. Using this approach, we analyzed review comments of three popular MOOCs on the Coursera Platform. The results indicate that the AHP approach is a feasible MOOC evaluation method that can provide accurate ratings as well as in-depth analysis of course design and learning outcomes. It is concluded that this new approach can supplement the existing user rating system with automated formative and summative evaluations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Waldrop, M.: Campus 2.0. Nature 495(7440), 160–163 (2013)

    Article  Google Scholar 

  2. Rhoads, R.A., Camacho, M.S., Toven-Lindsey, B., Lozano, J.B.: The massive open online course movement, xMOOCs, and faculty labor. Rev. High. Educ. 38(3), 397–424 (2015)

    Article  Google Scholar 

  3. Shapiro, H.B., Lee, C.H., Roth, N.E., Li, K., Çetinkaya-Rundel, M., Canelas, D.A.: Understanding the massive open online course (MOOC) student experience: an examination of attitudes, motivations, and barriers. Comput. Educ. 110, 35–50 (2017)

    Article  Google Scholar 

  4. Margaryan, A., Bianco, M., Littlejohn, A.: Instructional quality of massive open online courses (MOOCs). Comput. Educ. 80, 77–83 (2015)

    Article  Google Scholar 

  5. Pursel, B.K., Zhang, L., Jablokow, K.W., Choi, G.W., Velegol, D.: Understanding MOOC students: motivations and behaviours indicative of MOOC completion. J. Comput. Assist. Learn. 32(3), 202–217 (2016)

    Article  Google Scholar 

  6. Dick, W., Carey, C.: The Systematic Design of Instruction, 4th edn. Longman, New York (1996)

    Google Scholar 

  7. Parscal, T., Riemer, D.: Assuring quality in large-scale online course development. Online J. Distance Learn. Adm. 13(2), 7 (2010)

    Google Scholar 

  8. Wright, R.: Quality matters. Distance Learn. 15(3), 41–49 (1978)

    Google Scholar 

  9. Gillmore, G.M., Kane, M.T., Naccarato, R.W.: The generalizability of student ratings of instruction: estimation of the teacher and course components. J. Educ. Meas. 15(1), 1–13 (1978)

    Article  Google Scholar 

  10. Penn State Quality Assurance e-Learning Design Standards. https://weblearning.psu.edu/resources/penn-state-online-resources/penn-state-quality-assurance-e-learning-design-standards/. Accessed 16 Feb 2019

  11. Chua, A., Lam, W.: Quality assurance in online education: the universitas 21 global approach. Br. J. Edu. Technol. 38(1), 133–152 (2010)

    Article  Google Scholar 

  12. Saaty, T.L.: How to make a decision: the analytic hierarchy process. Eur. J. Oper. Res. 24(6), 19–43 (1994)

    Google Scholar 

  13. Lin, T.C., Ho, H.P., Chang, C.T.: Evaluation model for applying an e-learning system in a course: an analytic hierarchy process-multi-choice goal programming approach. J. Educ. Comput. Res. 50(1), 135–157 (2014)

    Article  Google Scholar 

  14. Lee, D., Mccool, J., Napieralski, L.: Assessing adult learning preferences using the analytic hierarchy process. Int. J. Lifelong Educ. 19(6), 548–560 (2000)

    Article  Google Scholar 

  15. Stracke, C.M.: The quality of MOOCs: how to improve the design of open education and online courses for learners? In: Zaphiris, P., Ioannou, A. (eds.) Learning and Collaboration Technologies. Novel Learning Ecosystems, LCT 2017. LNCS, vol. 10295, pp. 285–293. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58509-3_23

    Chapter  Google Scholar 

  16. Saaty, T.L., Vargas, L.G.: Models, Methods, Concepts & Applications of the Analytic Hierarchy Process. Springer, Boston (2001). https://doi.org/10.1007/978-1-4615-1665-1

    Book  MATH  Google Scholar 

  17. Wass, R., Golding, C.: Sharpening a tool for teaching: the zone of proximal development. Teach. High. Educ. 19(6), 671–684 (2014)

    Article  Google Scholar 

Download references

Acknowledgment

This study has been supported by the self-determined research funds of Central China Normal University from the colleges’ basic research and operation of MOE (CCNU19TD023) and the Basic Research Funding Grant of Central China Normal University (CCNU18QN023).

Author information

Authors and Affiliations

  1. School of Educational Information Technology, Central China Normal University, Wuhan, 430079, China

    Yanjiao Nie & Heng Luo

Authors
  1. Yanjiao Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heng Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Heng Luo .

Editor information

Editors and Affiliations

  1. The Open University of Hong Kong, Hong Kong, China

    Simon K. S. Cheung

  2. The Open University of Hong Kong, Hong Kong, China

    Lap-Kei Lee

  3. Jan Evangelista Purkyne University, ĂšstĂ­ nad Labem, Czech Republic

    Ivana Simonova

  4. University of Hradec Kralove, Hradec Kralove, Czech Republic

    Tomas Kozel

  5. City University of Hong Kong, Hong Kong, China

    Lam-For Kwok

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nie, Y., Luo, H. (2019). Diagnostic Evaluation of MOOCs Based on Learner Reviews: The Analytic Hierarchy Process (AHP) Approach. In: Cheung, S., Lee, LK., Simonova, I., Kozel, T., Kwok, LF. (eds) Blended Learning: Educational Innovation for Personalized Learning. ICBL 2019. Lecture Notes in Computer Science(), vol 11546. Springer, Cham. https://doi.org/10.1007/978-3-030-21562-0_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-21562-0_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-21561-3

  • Online ISBN: 978-3-030-21562-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation