Abstract
The DINA model is a commonly used model for obtaining diagnostic information. Like many other Diagnostic Classification Models (DCMs), it can require a large sample size to obtain reliable item and examinee parameter estimation. Neural Network (NN) analysis is a classification method that uses a training dataset for calibration. As a result, if this training dataset is determined theoretically, as was the case in Gierl’s attribute hierarchical method (AHM), the NN analysis does not have any sample size requirements. However, a NN approach does not provide traditional item parameters of a DCM or allow for item responses to influence test calibration. In this paper, the NN approach will be implemented for the DINA model estimation to explore its effectiveness as a classification method beyond its use in AHM. The accuracy of the NN approach across different sample sizes, item quality and Q-matrix complexity is described in the DINA model context. Then, a Markov Chain Monte Carlo (MCMC) estimation algorithm and Joint Maximum Likelihood Estimation is used to extend the NN approach so that item parameters associated with the DINA model are obtained while allowing examinee responses to influence the test calibration. The results derived by the NN, the combination of MCMC and NN (NN MCMC) and the combination of JMLE and NN are compared with that of the well-established Hierarchical MCMC procedure and JMLE with a uniform prior on the attribute profile to illustrate their strength and weakness.
Similar content being viewed by others
References
CASTEJÓN LIMAS, M., and ORDIERES MERÉ, J. (2009), “The AMORE Package: A MORE Flexible Neural Network Package” (R package), http://cran.r-project.org/.
DE LA TORRE, J. (2008), “An Empirically Based Method of Q-Matrix Validation for the DINA Model: Development and Applications”, Journal of Educational Measurement , 45(4), 343–362.
DE LA TORRE, J. (2009), “DINA Model and Parameter Estimation: A Didactic”, Journal of Educational and Behavioral Statistics, 34(1), 115–130.
DE LA TORRE, J., and DOUGLAS, J. (2004), “Higher Order Latent Trait Models for Cognitive Diagnosis, Psychometrika, 69(3), 333–353.
GARSON, G. (1998), Neural Networks: An Introductory Guide for Social Scientists, London: SAGE Publication Ltd.
GIERL, M. (2007), “Making Diagnostic Inferences about Cognitive Attributes Using the Rule-Space Model and Attribute Hierarchy Method”, Journal of Educational Measurement, 44(4), 325–340.
GIERL, M.J., Cui, Y., and HUNKA, S. (2007), “Using Connectionist Models to Evaluate Examinees’ Response Patterns on Tests”, paper presented at the annual meeting of the National Council on Measurement in Education, Chicago IL.
GIERL, M., WANG, C., and ZHOU, J. (2008), “Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees’ Cognitive Skills in Algebra on the SAT”, Journal of Technology, Learning, and Assessment, 6(6).
GIERL, M., ZHENG, Y., and CUI, Y. (2008), “Using the Attribute Hierarchy Method to Identify and Interpret Cognitive Skills that Produce Group Differences”, Journal of Educational Measurement, 45(1), 65–89.
HAERTEL, E. (1989), ``Using Restricted Latent Class Models to Map the Sill Structure of Achievement Items”, Journal of Educational Measurement (26(4), 301–321.
HENSON, R., and TEMPLIN, J. (2006), Implications of Q-matrix Misspecification in Cognitive Diagnosis, manuscript submitted for publication.
HENSON, R. (2008), “Functions of Estimating Log-Linear Cognitive Diagnostic Model,” Department of Educational Research Methodology, The University of North Carolina at Greensboro, Greensboro, NC.
JUNKER, B.W., and SIJTSMA, K. (2001), “Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory”, Applied Psychological Measurement, 25, 258–272.
LEIGHTON, J., GIERL, M., and HUNKA, S. (2004), “The Attribute Hierarchical Method for Cognitive Assessment: A Variation on Tatsuoka’s Rule-Space Approach”, Journal of Educational Measurement, 41(3), 205–237.
PATZ, R., and JUNKER, B. (1999), “A Straightforward Approach to Markov Chain Monte Carlo Methods for Item Response Models”, Journal of Educational and Behavioral Statistics, 24(2), 146–178.
RUPP, A., TEMPLIN, J., and HENSON, R. (2010), Diagnostic Measurement: Theory, Methods and Applications, New York, NY: The Guildford Press.
STERGIOU, C., and SIGANOS, D., “Neural Networks”, http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html.
TEMPLIN, J., HENSON, R., TEMPLIN, S., and ROUSSO, L. (2008), “Robustness of Hierarchical Modeling of Skill Association in Cognitive Diagnosis Models”, Applied Psychological Measurement (OnlineFirst).
TEMPLIN, J., and HENSON, R. (2006), ``Measurement of Psychological Disorders Using Cognitive Diagnosis Models”, Psychological Methods 11(3), 287–305.
WILLSE, J.T. (2010). “Functions for Estimating DINA and DINO Models Using JML or MML”, Department of Educational Research Methodology, The University of North Carolina at Greensboro, Greensboro, NC.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Shu, Z., Henson, R. & Willse, J. Using Neural Network Analysis to Define Methods of DINA Model Estimation for Small Sample Sizes. J Classif 30, 173–194 (2013). https://doi.org/10.1007/s00357-013-9134-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00357-013-9134-7