ABSTRACT
Personalized learner modeling using cognitive diagnosis (CD), which aims to model learners' cognitive states by diagnosing learner traits from behavioral data, is a fundamental yet significant task in many web learning services. Existing cognitive diagnosis models (CDMs) follow theproficiency-response paradigm that views learner traits and question parameters as trainable embeddings and learns them through learner performance prediction. However, we notice that this paradigm leads to the inevitable non-identifiability and explainability overfitting problem, which is harmful to the quantification of learners' cognitive states and the quality of web learning services. To address these problems, we propose an identifiable cognitive diagnosis framework (ID-CDF) based on a novelresponse-proficiency-response paradigm inspired by encoder-decoder models. Specifically, we first devise the diagnostic module of ID-CDF, which leverages inductive learning to eliminate randomness in optimization to guarantee identifiability and captures the monotonicity between overall response data distribution and cognitive states to prevent explainability overfitting. Next, we propose a flexible predictive module for ID-CDF to ensure diagnosis preciseness. We further present an implementation of ID-CDF, i.e., ID-CDM, to illustrate its usability. Extensive experiments on four real-world datasets with different characteristics demonstrate that ID-CDF can effectively address the problems without loss of diagnosis preciseness. Our code is available at https://github.com/CSLiJT/ID-CDF.
Supplemental Material
- Markus Bayer, Marc-André Kaufhold, and Christian Reuter. 2023. A Survey on Data Augmentation for Text Classification. ACM Comput. Surv., Vol. 55, 7 (2023), 146:1--146:39. https://doi.org/10.1145/3544558Google ScholarDigital Library
- George Casella Berger, Roger. 2024. Statistical Inference 2 ed.). Chapman and Hall/CRC, New York.Google Scholar
- Justyna Brzezinska. 2020. Item response theory models in the measurement theory. Commun. Stat. Simul. Comput., Vol. 49, 12 (2020), 3299--3313.Google ScholarCross Ref
- Lei Chen, Le Wu, Kun Zhang, Richang Hong, Defu Lian, Zhiqiang Zhang, Jun Zhou, and Meng Wang. 2023. Improving Recommendation Fairness via Data Augmentation. In Proceedings of the ACM Web Conference 2023 (Austin, TX, USA) (WWW '23). Association for Computing Machinery, New York, NY, USA, 1012--1020. https://doi.org/10.1145/3543507.3583341Google ScholarDigital Library
- Kyunghyun Cho, Bart van Merrienboer, cC aglar Gü lcc ehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In EMNLP. ACL, 1724--1734.Google Scholar
- Susan Craw. 2010. Manhattan Distance. Springer US, Boston, MA, 639--639. https://doi.org/10.1007/978-0--387--30164--8_506Google ScholarCross Ref
- Jimmy de la Torre. 2009. DINA Model and Parameter Estimation: A Didactic. Journal of Educational and Behavioral Statistics, Vol. 34, 1 (2009), 115--130.Google ScholarCross Ref
- Mingyu Feng, Neil T. Heffernan, and Kenneth R. Koedinger. 2009. Addressing the assessment challenge with an online system that tutors as it assesses. User Model. User Adapt. Interact., Vol. 19, 3 (2009), 243--266.Google ScholarDigital Library
- Gerhard H. Fischer. 1995. Derivations of the Rasch Model. Springer New York, New York, NY, 15--38.Google Scholar
- Francc ois Fouss, Alain Pirotte, Jean-Michel Renders, and Marco Saerens. 2007. Random-Walk Computation of Similarities between Nodes of a Graph with Application to Collaborative Recommendation. IEEE Trans. Knowl. Data Eng., Vol. 19, 3 (2007), 355--369.Google ScholarDigital Library
- Yingqiang Ge, Juntao Tan, Yan Zhu, Yinglong Xia, Jiebo Luo, Shuchang Liu, Zuohui Fu, Shijie Geng, Zelong Li, and Yongfeng Zhang. 2022. Explainable Fairness in Recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (Madrid, Spain) (SIGIR '22). Association for Computing Machinery, New York, NY, USA, 681--691. https://doi.org/10.1145/3477495.3531973Google ScholarDigital Library
- Alan E. Gelfand and Adrian F. M. Smith. 1990. Sampling-Based Approaches to Calculating Marginal Densities. J. Amer. Statist. Assoc., Vol. 85, 410 (1990), 398--409.Google ScholarCross Ref
- Golnaz Ghiasi, Yin Cui, Aravind Srinivas, Rui Qian, Tsung-Yi Lin, Ekin D. Cubuk, Quoc V. Le, and Barret Zoph. 2021. Simple Copy-Paste Is a Strong Data Augmentation Method for Instance Segmentation. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19--25, 2021. Computer Vision Foundation / IEEE, 2918--2928. https://doi.org/10.1109/CVPR46437.2021.00294Google ScholarCross Ref
- Mark Gierl and Jacqueline Leighton (Eds.). 2007. Cognitive Diagnostic Assessment for Education: Theory and Applications. Cambridge University Press, Cambridge. https://doi.org/10.1017/CBO9780511611186Google ScholarCross Ref
- Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In AISTATS (JMLR Proceedings, Vol. 9). JMLR.org, 249--256.Google Scholar
- W. K. Hastings. 1970. Monte Carlo Sampling Methods Using Markov Chains and Their Applications. Biometrika, Vol. 57, 1 (1970), 97--109.Google ScholarCross Ref
- Stamper J., Niculescu-Mizil A., Ritter S., G.J. Gordon, and Koedinger K.R. 2010. Algebra | 2006--2007. Development data set from KDD Cup 2010 Educational Data Mining Challenge. (2010). http://pslcdatashop.web.cmu.edu/KDDCup/downloads.jspGoogle Scholar
- Taegwan Kang, Hwanhee Lee, Byeongjin Choe, and Kyomin Jung. 2021. Entangled Bidirectional Encoder to Autoregressive Decoder for Sequential Recommendation. In SIGIR. ACM, 1657--1661.Google Scholar
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR (Poster).Google Scholar
- Sheng Li, Quanlong Guan, Liangda Fang, Fang Xiao, Zhenyu He, Yizhou He, and Weiqi Luo. 2022. Cognitive Diagnosis Focusing on Knowledge Concepts. In CIKM. ACM, 3272--3281.Google Scholar
- Xiaopeng Li and James She. 2017. Collaborative Variational Autoencoder for Recommender Systems. In KDD. ACM, 305--314.Google Scholar
- Qi Liu. 2021. Towards a New Generation of Cognitive Diagnosis. In IJCAI. ijcai.org, 4961--4964.Google Scholar
- Qi Liu, Run-ze Wu, Enhong Chen, Guandong Xu, Yu Su, Zhigang Chen, and Guoping Hu. 2018. Fuzzy Cognitive Diagnosis for Modelling Examinee Performance. ACM Trans. Intell. Syst. Technol., Vol. 9, 4 (2018), 48:1--48:26.Google ScholarDigital Library
- Jinwei Luo, Mingkai He, Weike Pan, and Zhong Ming. 2023. BGNN: Behavior-aware graph neural network for heterogeneous session-based recommendation. Frontiers of Computer Science, Vol. 17, 5 (12 Jan 2023), 175336. https://doi.org/10.1007/s11704-022--2100-yGoogle ScholarCross Ref
- Leland McInnes, John Healy, and James Melville. 2020. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arxiv: 1802.03426 [stat.ML]Google Scholar
- Qiong Nan, Juan Cao, Yongchun Zhu, Yanyan Wang, and Jintao Li. 2021. MDFEND: Multi-domain Fake News Detection. In CIKM. ACM, 3343--3347.Google Scholar
- Radek Pelánek. 2017. Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques. User Modeling and User-Adapted Interaction, Vol. 27, 3 (Dec. 2017), 313--350. https://doi.org/10.1007/s11257-017--9193--2Google ScholarDigital Library
- Shaina Raza and Chen Ding. 2022. Fake news detection based on news content and social contexts: a transformer-based approach. Int. J. Data Sci. Anal., Vol. 13, 4 (2022), 335--362.Google ScholarCross Ref
- Mark D. Reckase. 2009. Multidimensional Item Response Theory Models. Springer New York, New York, NY, 79--112.Google Scholar
- Suvash Sedhain, Aditya Krishna Menon, Scott Sanner, and Lexing Xie. 2015. AutoRec: Autoencoders Meet Collaborative Filtering. In WWW (Companion Volume). ACM, 111--112.Google ScholarDigital Library
- Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., Vol. 15, 1 (2014), 1929--1958.Google ScholarDigital Library
- Kikumi K. Tatsuoka. 1983. Rule Space: An Approach for Dealing with Misconceptions Based on Item Response Theory. Journal of Educational Measurement, Vol. 20, 4 (1983), 345--354.Google ScholarCross Ref
- Fei Wang, Qi Liu, Enhong Chen, Zhenya Huang, Yu Yin, Shijin Wang, and Yu Su. 2022. NeuralCD: A General Framework for Cognitive Diagnosis. IEEE Transactions on Knowledge and Data Engineering (2022), 1--16.Google Scholar
- Jinze Wu, Qi Liu, Zhenya Huang, Yuting Ning, Hao Wang, Enhong Chen, Jinfeng Yi, and Bowen Zhou. 2021. Hierarchical Personalized Federated Learning for User Modeling. In Proceedings of the Web Conference 2021 (Ljubljana, Slovenia) (WWW '21). Association for Computing Machinery, New York, NY, USA, 957--968. https://doi.org/10.1145/3442381.3449926Google ScholarDigital Library
- Lianwei Wu, Yuan Rao, Cong Zhang, Yongqiang Zhao, and Ambreen Nazir. 2023. Category-Controlled Encoder-Decoder for Fake News Detection. IEEE Trans. Knowl. Data Eng., Vol. 35, 2 (2023), 1242--1257.Google Scholar
- Mike Wu, Richard Lee Davis, Benjamin W. Domingue, Chris Piech, and Noah D. Goodman. 2020. Variational Item Response Theory: Fast, Accurate, and Expressive. In EDM. International Educational Data Mining Society.Google Scholar
- Yao Wu, Christopher DuBois, Alice X. Zheng, and Martin Ester. 2016. Collaborative Denoising Auto-Encoders for Top-N Recommender Systems. In WSDM. ACM, 153--162.Google Scholar
- Gongjun Xu. 2019. Identifiability and Cognitive Diagnosis Models. Springer International Publishing, Cham, 333--357.Google Scholar
- Gongjun Xu and Stephanie Zhang. 2016. Identifiability of Diagnostic Classification Models. Psychometrika, Vol. 81, 3 (Sept. 2016), 625--649. https://doi.org/10.1007/s11336-015--9471-zGoogle ScholarCross Ref
- Peng Xu and Michel C. Desmarais. 2018. An Empirical Research on Identifiability and Q-matrix Design for DINA model. In EDM. International Educational Data Mining Society (IEDMS).Google Scholar
- Chun-Kit Yeung. 2019. Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory. In EDM. International Educational Data Mining Society (IEDMS).Google Scholar
- Shengjun Yin, Kailai Yang, and Hongzhi Wang. 2020. A MOOC Courses Recommendation System Based on Learning Behaviours. In ACM TUR-C'20: ACM Turing Celebration Conference, Hefei, China, May 22--24, 2020. ACM, 133--137. https://doi.org/10.1145/3393527.3393550Google ScholarDigital Library
- Jifan Yu, Yuquan Wang, Qingyang Zhong, Gan Luo, Yiming Mao, Kai Sun, Wenzheng Feng, Wei Xu, Shulin Cao, Kaisheng Zeng, Zijun Yao, Lei Hou, Yankai Lin, Peng Li, Jie Zhou, Bin Xu, Juanzi Li, Jie Tang, and Maosong Sun. 2021. MOOCCubeX: A Large Knowledge-Centered Repository for Adaptive Learning in MOOCs. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (Virtual Event, Queensland, Australia) (CIKM '21). Association for Computing Machinery, New York, NY, USA, 4643--4652. https://doi.org/10.1145/3459637.3482010Google ScholarDigital Library
- Chuang Zhao, Hongke Zhao, Ming HE, Jian Zhang, and Jianping Fan. 2023. Cross-domain recommendation via user interest alignment. In Proceedings of the ACM Web Conference 2023 (Austin, TX, USA) (WWW '23). Association for Computing Machinery, New York, NY, USA, 887--896.Google ScholarDigital Library
Index Terms
- Towards the Identifiability and Explainability for Personalized Learner Modeling: An Inductive Paradigm
Recommendations
Towards Comprehensive User Modeling on the Social Web for Personalized Link Recommendations
UMAP '16: Proceedings of the 2016 Conference on User Modeling Adaptation and PersonalizationUser modeling for individual users on the Social Web plays a significant role and is a fundamental step for personalization as well as recommendations. Previous studies have proposed various user modeling strategies in different dimensions such as (1) ...
Continuous Personalized Knowledge Tracing: Modeling Long-Term Learning in Online Environments
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge ManagementWith the advance of online education systems, accessibility to learning materials has increased. In these systems, students can practice independently and learn from different learning materials over long periods of time. As a result, it is essential to ...
Augmenting Personalized Question Recommendation with Hierarchical Information for Online Test Platform
Advanced Data Mining and ApplicationsAbstractPersonalized question recommendation for students is an important research topic in the field of smart education. Current studies depend on collaborative filtering based, cognitive diagnosis based, or cognitive diagnosis based on collaborative ...
Comments