A case of the inapplicability of the Rasch model: Mapping conceptual learning

Abstract

The basic theory of Rasch measurement applies to situations where a person has a certain level of a trait being investigated, and this level of ability is what determines (to within a measurement error) how well the person does on each item in a test. This paper responds to frequent suggestions from colleagues that the use of Rasch measurement would be profitable in analysing a set of data on students’ understanding of decimal notation. We demonstrate misfit to the Rasch model by showing that item difficulty estimates show important variation by year level, that there is significant deviation from expected score curves, and that success on certain splitter items does not imply a student is more likely to score well on other items. The explanation given is that conceptual learning may not always be able to be measured on a scale, which is an essential feature of the Rasch approach. Instead, students move between categories of interpretations, which do not necessarily provide more correct answers even when they are based on an improved understanding of fundamental principles. In this way, the paper serves to highlight the assumptions built into the Rasch model and to discuss its applicability to describing the progress of learning with various characteristics.