Abstract
This paper presents exploratory research regarding the design and evaluation of a dashboard supporting the advising of aspiring university students incorporating a black-box predictive model for student success. While black-box predictive models can provide accurate predictions, incorporating them in dashboards is challenging as the black-box nature can threaten the interpretability and negatively impact trust of end-users. Explainable Learning Analytics aims to provide insights to black-box predictions by for instance explaining how the input features impact the prediction made. Two dashboards were designed to visualize the prediction and the outcome of the explainer. The dashboards supplemented the explainer with an interactive visualisation allowing to simulate how changes in the student’s features impact the prediction. Both dashboards were evaluated in user tests with 13 participants. The results show the potential of explainable AI techniques to bring predictive models to advising practice. We found that the combination of the explainer with the simulation helped users to compare the predictive model with their mental models of student success, challenging understanding of users and influencing trust in the predictive model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Due to Flemish regulations, no data on prior academic career is transferred from secondary to higher education.
References
Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on eXplainable Artificial Intelligence (XAI). IEEE Access 6, 52138–52160 (2018). https://doi.org/10.1177/0149206321997910
Al-Sudani, S., Palaniappan, R.: Predicting students’ final degree classification using an extended profile. Educ. Inf. Technol. 24(4), 2357–2369 (2019). https://doi.org/10.1007/s10639-019-09873-8
Alamri, R., Alharbi, B.: Explainable student performance prediction models: a systematic review. IEEE Access 9, 33132–33143 (2021). https://doi.org/10.1109/ACCESS.2021.3061368
Brooke, J.: SUS - a quick and dirty usability scale (1996)
Broos, T., Pinxten, M., Margaux, D., Verbert, K., De Laet, T.: Learning dashboards at scale: early warning and overall first year experience. Assess. Eval. High. Educ. 45(6), 855–874 (2020). https://doi.org/10.1080/02602938.2019.1689546
Charleer, S., Moere, A.V., Klerkx, J., Verbert, K., De Laet, T.: Learning analytics dashboards to support adviser-student dialogue. IEEE Trans. Learn. Technol. 11(3), 389–399 (2018). https://doi.org/10.1109/TLT.2017.2720670
Davis, B., Glenski, M., Sealy, W., Arendt, D.: Measure utility, gain trust: practical advice for XAI researchers. In: 2020 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX), pp. 1–8 (2020). https://doi.org/10.1109/TREX51495.2020.00005
De Laet, T., Millecamp, M., Ortiz-Rojas, M., Jimenez, A., Maya, R., Verbert, K.: Adoption and impact of a learning analytics dashboard supporting the advisor-student dialogue in a higher education institute in Latin America. BJET 51(4), 1002–1018 (2020). https://doi.org/10.1111/bjet.12962
Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine learning. arXiv e-prints arXiv:1702.08608 (2017)
Essa, A., Ayad, H.: Student success system: risk analytics and data visualization using ensembles of predictive models. In: Proceedings of 2nd International Conference on Learning Analytics and Knowledge, LAK 2012, pp. 158–161. Association for Computing Machinery, New York (2012). https://doi.org/10.1145/2330601.2330641
Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., Kagal, L.: Explaining explanations: an overview of interpretability of machine learning. In: 2018 IEEE 5th International Conference on Data Science and Advanced Analytics, pp. 80–89 (2018). https://doi.org/10.1109/DSAA.2018.00018
Gutiérrez, F., Seipp, K., Ochoa, X., Chiluiza, K., De Laet, T., Verbert, K.: LADA: a learning analytics dashboard for academic advising. Comput. Hum. Behav. 107, 105826 (2020). https://doi.org/10.1016/j.chb.2018.12.004
Hilliger, I., et al.: For learners, with learners: identifying indicators for an academic advising dashboard for students. In: Alario-Hoyos, C., Rodríguez-Triana, M.J., Scheffel, M., Arnedillo-Sánchez, I., Dennerlein, S.M. (eds.) EC-TEL 2020. LNCS, vol. 12315, pp. 117–130. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-57717-9_9
Kommiya Mothilal, R.: Statistical modeling of students’ performance in an open-admission bachelor program in Flanders (2018)
van Leeuwen, A.: Learning analytics to support teachers during synchronous CSCL: balancing between overview and overload. J. Learn. Anal. 2(2), 138–162 (2015). https://doi.org/10.18608/jla.2015.22.11
Pinxten, M., Van Soom, C., Peeters, C., De Laet, T., Langie, G.: At-risk at the gate: prediction of study success of first-year science and engineering students in an open-admission university in Flanders—any incremental validity of study strategies? Eur. J. Psychol. Educ. 34(1), 45–66 (2017). https://doi.org/10.1007/s10212-017-0361-x
Ribeiro, M., Singh, S., Guestrin, C.: “Why should i trust you?”: explaining the predictions of any classifier. In: Proceedings of 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations. Association for Computational Linguistics (2016). https://doi.org/10.18653/v1/n16-3020
Scheffel, M.: Evaluation framework for LA (2017). http://www.laceproject.eu/evaluation-framework-for-la/. Accessed 06 May 2020
Schmitz, M., van Limbeek, E., Greller, W., Sloep, P., Drachsler, H.: Opportunities and challenges in using learning analytics in learning design. In: Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M. (eds.) EC-TEL 2017. LNCS, vol. 10474, pp. 209–223. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66610-5_16
Spinner, T., Schlegel, U., Schafer, H., El-Assady, M.: explAIner: a visual analytics framework for interactive and explainable machine learning. IEEE Trans. Vis. Comput. Graph. 1 (2019). https://doi.org/10.1109/tvcg.2019.2934629
Vanderoost, J., et al.: Engineering and science positioning tests in Flanders: powerful predictors for study success? In: Proceedings of 43rd SEFI Conference 2015, pp. 1–8. SEFI, Brussels (2015)
Weinstein, C.E., Palmer, D.R.: LASSI - Learning and Study Strategies Inventory. 2nd edn. (2002). www.collegelassi.com/lassi/. Accessed 22 May 2020
Yin, M., Wortman Vaughan, J., Wallach, H.: Understanding the effect of accuracy on trust in machine learning models. In: Proceedings of 2019 CHI Conference, pp. 1–12. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3290605.3300509
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Scheers, H., De Laet, T. (2021). Interactive and Explainable Advising Dashboard Opens the Black Box of Student Success Prediction. In: De Laet, T., Klemke, R., Alario-Hoyos, C., Hilliger, I., Ortega-Arranz, A. (eds) Technology-Enhanced Learning for a Free, Safe, and Sustainable World. EC-TEL 2021. Lecture Notes in Computer Science(), vol 12884. Springer, Cham. https://doi.org/10.1007/978-3-030-86436-1_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-86436-1_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86435-4
Online ISBN: 978-3-030-86436-1
eBook Packages: Computer ScienceComputer Science (R0)