Abstract
Online learning is gradually becoming a popular way of learning due to the high flexibility in time and space. Reducing the high dropout rate is important to promote the further development of smart education. However, learners’ learning is a dynamic temporal process, which is influenced by multiple factors synergistically. How to identify the key influencing factors of dropout in an interpretable way is still a challenging problem. In this paper, we propose a pattern identification method of dropout behavior, including the prediction of the dropout probability and the mining of potential impact factors, to gain a comprehensive insight into the dropout behavior hidden in the data. A CNN-LSTM model for dropout prediction is constructed, which can automatically extract features and learn the temporal dependence of dropout behavior. By introducing the counterfactual explanation, the dropout impacts of different learning behavior can be revealed quantitatively. Moreover, we design and develop an interactive visual analytics system, DropoutVis, for exploring learning behavior, extracting the various dropout patterns and providing a basis for formulating strategies. The effectiveness and usefulness of DropoutVis have been demonstrated through case studies with a real dataset.
Graphical abstract
Similar content being viewed by others
References
Amnueypornsakul B, Bhat S, Chinprutthiwong P (2014) Predicting attrition along the way: the UIUC model. In: Proceedings of the EMNLP 2014 workshop on analysis of large scale social interaction in MOOCs, pp 55–59
Chen Q, Chen Y, Liu D, Shi C, Wu Y, Qu H (2016) Peakvizor: visual analytics of peaks in video clickstreams from massive open online courses. IEEE Trans Visual Comput Graph 22(10):2315–2330
Chen Q, Yue X, Plantaz X, Chen Y, Shi C, Pong T, Qu H (2020) Viseq: visual analytics of learning sequence in massive open online courses. IEEE Trans Visual Comput Graph 26(3):1622–1636
Chen Y,Chen Q, Zhao M, Boyer S, Veeramachaneni K, Qu H (2016) Dropoutseer: visualizing learning patterns in massive open online courses for dropout reasoning and prediction. In: 11th IEEE conference on visual analytics science and technology, IEEE VAST 2016, Baltimore, MD, USA, October 23–28, 2016. IEEE Computer Society, pp 111–120
Cheng F, Ming Y, Qu H (2021) DECE: decision explorer with counterfactual explanations for machine learning models. IEEE Trans Visual Comput Graph 27(2):1438–1447
Fisnik D, Shariq IA, Zenun K (2018) Mooc dropout prediction using machine learning techniques: review and research challenges. In 2018 IEEE global engineering education conference (EDUCON), pp 1007–1014
Fu S, Wang Y, Yang Y, Bi Q, Guo F, Qu H (2018) Visforum: A visual analysis system for exploring user groups in online forums. ACM Trans Interact Intell Syst 8(1):3:1-3:21
Fu S, Zhao J, Cui W, Qu H (2017) Visual analysis of MOOC forums with iForum. IEEE Trans Visual Comput Graph 23(1):201–210
Guidotti R, Monreale A, Turini F, Pedreschi D, Giannotti F (2018) A survey of methods for explaining black box models. ACM Comput Surv 51(5):1–42
Hohman F, Park H, Robinson C, Chau DHP (2020) Summit: Scaling deep learning interpretability by visualizing activation and attribution summarizations. IEEE Trans Visual Comput Graph 26(1):1096–1106
Huijie Z, Ren K, Yiming L, Dezhan Q, Li Z (2019) Airinsight: visual exploration and interpretation of latent patterns and anomalies in air quality data. Sustainability 11(10):2944
Jiazhi X, Jie L, Siming C, Hongxing Q, Shixia L (2021) A survey on interdisciplinary research of visualization and artificial intelligence. Sci Sin Inf 51:1777–1801
Jin C (2020) Mooc student dropout prediction model based on learning behavior features and parameter optimization. Interact Learn Environ 1–19. https://doi.org/10.1080/10494820.2020.1802300
Jinghan B, Huijie Z, Dezhan Q, Cheng L, Weizhang S (2021) Fgvis: visual analytics of human mobility patterns and urban areas based on f-glove. J Visual 24:1319–1335
Kahng M, Andrews PY, Kalro A, Chau DHP (2018) Activis: Visual exploration of industry-scale deep neural network models. IEEE Trans Visual Comput Graph 24(1):88–97
Chiu TKF, Hew TKF (2018) Factors influencing peer learning and performance in mooc asynchronous online discussion forum. Austral J Educ Technol 34(4):16–28
Kőrösi G, Farkas R (2020) Mooc performance prediction by deep learning from raw clickstream data. In: International conference on advances in computing and data sciences. Springer, Berlin, pp 474–485
Liu M, Shi J, Cao K, Zhu J, Liu S (2018) Analyzing the training processes of deep generative models. IEEE Trans Visual Comput Graph 24(1):77–87
Mehmet S (2021) A comparative analysis of dropout prediction in massive open online courses. Arab J Sci Eng 46:1845–1861
Ming Y, Cao S, Zhang R, Li Z, Chen Y, Song Y, Qu H (2017) Understanding hidden memories of recurrent neural networks. In: 12th IEEE conference on visual analytics science and technology, IEEE VAST 2017, Phoenix, AZ, USA, Oct 3–6, 2017. IEEE Computer Society, pp 13–24
Mothilal RK, Sharma A, Tan C (2020) Explaining machine learning classifiers through diverse counterfactual explanations. In: Proceedings of the 2020 conference on fairness, accountability, and transparency, pp 607–617
Mu X, Xu K, Chen Q, Du F, Wang Y, Qu H (2019) Moocad: visual analysis of anomalous learning activities in massive open online courses. In: 21st eurographics conference on visualization, EuroVis 2019—short papers, Porto, Portugal, June 3–7, 2019. Eurographics Association, pp 91–95
Pawelczyk M, Broelemann K, Kasneci G (2020) Learning model-agnostic counterfactual explanations for tabular data. In: Proceedings of the web conference 2020, pp 3126–3132
Prashan M, Tim M, Liz S, Frank V (2020) Explainable reinforcement learning through a causal lens. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 2493–2500, 04
Qian F, Zhanghao G, Junyi Z, Yafeng Z (2021) CLSA: a novel deep learning model for MOOC dropout prediction. Comput Electr Eng 94:107315
Reuben B, Max VK, Michael V, Ulrik L, Jun Z, Nigel S (2018) ’It’s reducing a human being to a percentage’: perceptions of justice in algorithmic decisions. In: Proceedings of the 2018 CHI conference on human factors in computing systems, pp 1–14
Siby P, Manish G (2020) Onet c a temporal meta embedding network for MOOC dropout prediction. In: 2018 IEEE global engineering education conference (EDUCON), pp 5209–5217, 12
Strobelt H, Gehrmann S, Pfister H, Rush AM (2018) LSTMVIS: a tool for visual analysis of hidden state dynamics in recurrent neural networks. IEEE Trans Visual Comput Graph 24(1):667–676
Sun Z, Harit A, Yu J, Cristea AI, Shi L (2021) A brief survey of deep learning approaches for learning analytics on MOOCs. In: International conference on intelligent tutoring systems. Springer, Berlin, pp 28–37
Tharindu PP, Liyanagunawardena R, Williams S (2014) Dropout: Mooc participants perspective. pp 95–100, 02
Tulio RM, Sameer S, Carlos G (2016) “Why should I trust you?” Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1135–1144, 08
Verma S, Dickerson J, Hines K (2020) Counterfactual explanations for machine learning: a review. arXiv preprint arXiv:2010.10596
Wachter S, Mittelstadt B, Russell C (2017) Counterfactual explanations without opening the black box: automated decisions and the GDPR. JOLT 31:841
Wang L, Wang H(2019) Learning behavior analysis and dropout rate prediction based on MOOCs data. In: 2019 10th international conference on information technology in medicine and education (ITME). IEEE, pp 419–423
Wang W, Yu H, Miao C (2017) Deep model for dropout prediction in MOOCs. In: Proceedings of the 2nd international conference on crowd science and engineering, pp 26–32
Wangli X, Dongping D (2018) Dropout prediction in MOOCs: using deep learning for personalized intervention. J Educ Comput Res 57:073563311875701
Wen Y, Tian Y, Wen B, Zhou Q, Cai G, Liu S (2019) Consideration of the local correlation of learning behaviors to predict dropouts from MOOCs. Tsinghua Sci Technol 25(3):336–347
Wenzheng F, Jie T, Tracy L (2019) Understanding dropouts in MOOCs. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 517–524, 07
Wong J-S et al (2018) Messagelens: a visual analytics system to support multifaceted exploration of MOOC forum discussions. Visual Inform 2(1):37–49
Wu N, Zhang L, Gao Y, Zhang M, Sun X, Feng J (2019) CLMS-Net: dropout prediction in MOOCs with deep learning. In: Proceedings of the ACM turing celebration conference, China, pp 1–6
Xia M, Sun M, Wei H, Chen Q, Wang Y, Shi L, Qu H, Ma X (2019) Peerlens: peer-inspired interactive learning path planning in online question pool. In: Proceedings of the 2019 CHI conference on human factors in computing systems, CHI 2019, Glasgow, Scotland, UK, May 04-09. ACM, p 634
Xia M, Velumani RP, Wang Y, Qu H, Ma X (2021) Qlens: visual analytics of multi-step problem-solving behaviors for improving question design. IEEE Trans Visual Comput Graph 27(2):870–880
Xia M, Xu M, Lin C, Cheng TY, Qu H, Ma X (2020) Seqdynamics: visual analytics for evaluating online problem–solving dynamics. Comput Graph Forum 39(3):511–522
Xiong F, Zou K, Liu Z, Wang H (2019) Predicting learning status in MOOCs using LSTM. In: Proceedings of the ACM turing celebration conference, China, pp 1–5
Xueyu W, Gang Z, Xiao L (2017) Research on the learners’ dropout prediction based on the MOOC data. Mod Educ Technol 27(06):94–100
Ye C , Biswas G (2014) Early prediction of student dropout and performance in MOOCs using higher granularity temporal information. J Learn Anal 1(3):169–172
Yonggu W, Qing Z (2014) MOOC: characteristics and learning mechanism. Educ Res 35(09):112–120133
Yu C-H, Wu J, Liu A-C (2019) Predicting learning outcomes with MOOC clickstreams. Educ Sci 9(2):104
Zaporozhko VV, Parfenov DI Shardakov VM (2019) Development approach of formation of individual educational trajectories based on neural network prediction of student learning outcomes. In: International conference of artificial intelligence, medical engineering, education. Springer, Berlin, pp 305–314
Zhao Y, Shi J, Liu J, Zhao J, Zhou F, Zhang W, Chen K, Zhao X, Zhu C, Chen W (2021) Evaluating effects of background stories on graph perception. IEEE Trans Visual Comput Graph 28:4839–4854
Zhenguo X, Guanwen Z, Lin S, Jing A (2017) Research on the factors affecting the MOOC learner’ dropout behavior. Mod Educ Technol 27(09):100–106
Acknowledgements
This work was supported by National Natural Science Foundation of China under Grant 42171450, the Key Research and Development Project of Science and Technology Development Plan of Science and Technology Department of Jilin Province No. 20210201074GX, National Natural Science Foundation of China under Grant 41671379 and National Key R&D Program of China No. 2020YFA0714102.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhang, H., Dong, J., Lv, C. et al. Visual analytics of potential dropout behavior patterns in online learning based on counterfactual explanation. J Vis 26, 723–741 (2023). https://doi.org/10.1007/s12650-022-00899-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12650-022-00899-8