Context and Inference in Language Testing

Abstract

It is arguably the case that ‘The purpose of language testing is always to render information to aid in making intelligent decisions about possible courses of action’ (Carroll, 1961, p. 314). This holds true whether the decisions are primarily pedagogic, or affect the future education or employment of the test taker. If fair and useful decisions are to be made, three conditions must hold. Firstly, valid inferences must be made about the meaning of test scores. Secondly, score meaning must be relevant and generalisable to a real-world domain. Thirdly, score meaning should be (at least partially) predictive of post-decision performance. If any of these conditions are not met, the process of assessment and decision-making may be questioned not only in theory but also in the courts (Fulcher, 2014a). It is therefore not surprising that historically, testing practice has rested on the assumption that language competence, however defined, is a relatively stable cognitive trait. This is expressed clearly in classic statements of the role of measurement in the ‘human sciences’, such as this by father of American psychology James McKeen Cattell (1890, p. 380):

One of the most important objects of measurement … is to obtain a general knowledge of the capacities of a man by sinking shafts, as it were, at a few critical points. In order to ascertain the best points for the purpose, the sets of measures should be compared with an independent estimate of the man’s powers. We thus may learn which of the measures are the most instructive.