Sam Saarinen
ABSTRACT
Modeling student knowledge is important for assessment design, adaptive testing, curriculum design, and pedagogical intervention. The assessment design community has primarily focused on continuous latent-skill models with strong conditional independence assumptions among knowledge items, while the prerequisite discovery community has developed many models that aim to exploit the interdependence of discrete knowledge items. This paper attempts to bridge the gap by asking, "When does modeling assessment item interdependence improve predictive accuracy?" A novel adaptive testing evaluation framework is introduced that is amenable to techniques from both communities, and an efficient algorithm, Directed Item-Dependence And Confidence Thresholds (DIDACT), is introduced and compared with an Item-Response-Theory based model on several real and synthetic datasets. Experiments suggest that assessments with closely related questions benefit significantly from modeling item interdependence.
- Giovanni Adorni, Chiara Alzetta, Frosina Koceva, Samuele Passalacqua, and Ilaria Torre. 2019. Towards the Identification of Propaedeutic Relations in Textbooks. In International Conference on Artificial Intelligence in Education. Springer, 1--13.Google Scholar
- Jaroslav Bayer, Hana Bydžovská, and Jan Géryk. 2012. Towards course prerequisites refinement. IMEA 2012 (2012), 4.Google Scholar
- Anthony F Botelho, Seth A Adjei, and Neil T Heffernan. 2016. Modeling Interactions across Skills: A Method to Construct and Compare Models Predicting the Existence of Skill Relationships. International Educational Data Mining Society (2016).Google Scholar
- Emma Brunskill. 2011. Estimating Prerequisite Structure From Noisy Data.. In EDM. Citeseer, 217--222.Google Scholar
- R Philip Chalmers et al. 2016. Generating adaptive and non-adaptive test interfaces for multidimensional item response theory applications. Journal of Statistical Software 71, 5 (2016), 1--39.Google Scholar
- Haw-Shiuan Chang, Hwai-Jung Hsu, and Kuan-Ta Chen. 2015. Modeling Exercise Relationships in E-Learning: A Unified Approach.. In EDM. 532--535.Google Scholar
- Yetian Chen, José P González-Brenes, and Jin Tian. 2016. Joint Discovery of Skill Prerequisite Graphs and Student Models. International Educational Data Mining Society (2016).Google Scholar
- Yang Chen, Pierre-Henr Wuillemin, and Jean-Marc Labat. 2015. Discovering Prerequisite Structure of Skills through Probabilistic Association Rules Mining. International Educational Data Mining Society (2015).Google Scholar
- Michel Desmarais et al. 2011. Conditions for effectively deriving a q-matrix from data with non-negative matrix factorization. In 4th international conference on educational data mining, EDM. 41--50.Google Scholar
- Soo-Yun Han, Jiyoung Yoon, and Yun Joo Yoo. 2017. Discovering skill prerequisite structure through Bayesian estimation and nested model comparison.. In EDM.Google Scholar
- Pan Liao, Yuan Sun, Shiwei Ye, Xin Li, Guiping Su, and Yi Sun. 2017. Predicting learners' multi-question performance based on neural networks. In 2017 International Conference on Behavioral, Economic, Socio-cultural Computing (BESC). IEEE, 1--6.Google ScholarCross Ref
- Danny Lynch and Colm P Howlin. 2014. Real world usage of an adaptive testing algorithm to uncover latent knowledge. In 7th international conference of education, research and innovation (ICERI2014 proceedings). IATED, Seville, Spain. 504--511.Google Scholar
- Sein Minn, Yi Yu, Michel C. Desmarais, Feida Zhu, and Jill Jenn Vie. 2018. Deep Knowledge Tracing and Dynamic Student Classification for Knowledge Tracing. Proceedings - IEEE International Conference on Data Mining, ICDM 2018-Novem (2018), 1182--1187. arXiv:arXiv:1809.08713v1 Google ScholarCross Ref
- Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas Guibas, and Jascha Sohl-Dickstein. 2015. Deep knowledge tracing. Advances in Neural Information Processing Systems 2015-Janua (2015), 505--513.Google Scholar
- Martin Plajner. 2016. Student Skill Models in Adaptive Testing. Journal of Machine Learning Research 52 (2016), 403--414. http://www.jmlr.org/proceedings/papers/v52/plajner16.htmlGoogle Scholar
- Martin Plajner. 2017. Probabilistic Models for Computerized Adaptive Testing. (2017). arXiv:1703.09794 http://arxiv.org/abs/1703.09794Google Scholar
- Sam Saarinen, Shriram Krishnamurthi, Kathi Fisler, and Preston Tunnell Wilson. 2019. Harnessing the Wisdom of the Classes: Classsourcing and Machine Learning for Assessment Instrument Generation. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. ACM, 606--612.Google ScholarDigital Library
- Jill Jenn Vie. 2016. Adaptive Testing Using a General Diagnostic Model. 2 (2016), 640--643. Google ScholarCross Ref
- Jill-Jênn Vie. 2018. Deep Factorization Machines for Knowledge Tracing. (2018), 370--373. arXiv:arXiv:1811.03388v2 Google ScholarCross Ref
- David J Weiss. 1985. Adaptive testing by computer. Journal of consulting and clinical psychology 53, 6 (1985), 774.Google ScholarCross Ref
- Runze Wu, Qi Liu, Yuping Liu, Enhong Chen, Yu Su, Zhigang Chen, and Guoping Hu. 2015. Cognitive modelling for predicting examinee performance. In Twenty-Fourth International Joint Conference on Artificial Intelligence.Google Scholar
Index Terms
- Applying prerequisite structure inference to adaptive testing
Recommendations
An experimental study of adaptive testing for software reliability assessment
Adaptive testing is a new form of software testing that is based on the feedback and adaptive control principle and can be treated as the software testing counterpart of adaptive control. Our previous work has shown that adaptive testing can be ...
Creating personalized assessments based on learner knowledge and objectives in a hypermedia Web testing application
This paper presents an adaptive Web testing system where the adaptation of the testing procedure relies on the performance, the prior knowledge and the goals and preferences of the test participants. The main aim of the paper is to discuss the utility ...
Evaluation of sequential adaptive testing with real-data simulation: A case study
Soft computing and intelligent systems: Tools, techniques and applicationsComputer-based testing systems take advantage of the interaction between computers and individuals to sequentially customize the presented test items to the test-taker’s ability estimate. Administering such sequential adaptive tests has many benefits ...
Comments