Singaporean teachers’ views of classroom assessment: Findings from using Q-methodology

Singaporean teachers’ views of classroom assessment: Findings from using Q-methodology

Wei Shin Leong

Abstract

This article details the use of Q methodology (Q) for eliciting different views of classroom assessment among Singaporean teachers. Q is purported to be useful for studying the tacit decision-making process within the wide-ranging practical and controversial situations that Q researchers tend to prefer to investigate. This article asks a key question: Is Q helpful for investigating the possible multiplicity and significant relationships of views of classroom assessment when new assessment policies are being introduced in Singaporean schools? The different clusters of viewpoints from the Q-factor analysis, in particular, reveal underlying bases of common and different espoused practices of classroom assessment. This suggests opportunities for consensus-building that will be useful for teachers, policy-makers, professional-learning developers, and researchers.

Recognising that certainty is not absolute in the twenty-first century, education authorities in some Asian countries have introduced policies that attempt to focus on students’ holistic development rather than just academic excellence (Curriculum Development Council (Hong Kong), 2001; PERI; 2009). The review and implementation of classroom assessment policy changes are quite different from those of the rest of the world, as the students from countries such as China (Shanghai), Singapore, Hong Kong, and Korea are already consistently “top of the class” in international comparative measures of educational achievement (McKinsey, 2007; OECD Newsroom, 2013; World Economic Forum, 2009). Supporting schools and teachers to review and widen the scope of classroom assessment practices seems like a logical “next step” to ensure that administrative, curriculum, and assessment innovations can keep pace with one another in terms of the recognition of more widely valued learning outcomes and processes. Such radical reviews and changes, according to researchers such as Hogan et al. (2013), cannot find their lineage in educational systems in countries where the results of international comparative measures of educational achievement were seen as inadequate, and have tightened the standardised testing regimes. In this respect, many researchers (e.g., Brown, Kennedy, Fok, Kin, & Yu, 2009) have pointed out that issues of assessment are not just a matter of enhancing assessment literacy of teachers, but require thorough understanding of broad social contexts that may not be easily recognisable and interpretable even by local observers.

Many researchers have also warned that the embedding of innovative classroom assessment practices originating from Anglophone countries, such as authentic assessment (Koh & Luke, 2009), formative assessment, and Assessment for Learning (AfL), has generally yielded very uneven outcomes (Berry, 2011; Black & Wiliam, 2005). Berry (2011) and other researchers have highlighted the need to be sensitive to existing indigenised values and practices in Asian classrooms (Brown et al., 2009), as formative assessment and AfL are essentially Western constructs (Kennedy, Chan, Fok, & Yu, 2008; Stobart, 2012). They have argued that adopting new classroom assessment practices cannot be thought of simply as meeting certain technical standards and expectations that have their roots in classrooms in Anglophone countries. Rather, classroom assessment practices are deeply complex social processes that cannot be easily “transplanted” from one culture to another (Carless 2005, 2011), if indeed it is even deemed useful and workable in those classrooms in the first place. Therefore, the notion that a particular classroom assessment can be considered innovative in a self-evident way or guaranteed by a global theory is extremely problematic. This study seeks to expand a very limited set of studies on how a particular Asian country is starting to undergo a reconceptualisation journey of classroom assessment. The study is premised upon the belief that sustainable and real changes can only be made by first enabling teachers to know their own views of classroom assessment in the first place. In doing this, we can begin to reconsider practices that are neither constrained by the presuppositions of conventional thinking nor necessarily foreclosed by the exigencies of existing practice.

Contextual background

Following a review of primary-school education in 2009, the Singapore government supported a key recommendation by the Primary School Review Committee (PERI) to address the overemphasis on testing and examination, particularly at lower primary levels. PERI recommended that “holistic assessment” which supports student learning would be progressively introduced in all primary-school classrooms, starting with lower primary in 2011 (PERI, 2009). At the same time, another assessment review committee within the Ministry of Education was convened to review and explore ways to refine the examination and assessment landscape across all other Singaporean schools. The recommendation for changes in assessment beyond primary schools, as proposed by the committee, involved helping secondary and post-secondary teachers to think about the possibilities of “balanced assessment” involving the judicious use of formative and summative assessment.

Considering the output of policy documents and research literature on classroom assessment that have emerged around the world since the 1990s (e.g., Black & Wiliam, 1998, 2005), education initiatives that support the use of assessment to enhance teaching and learning in the classroom are a relatively late development. One might speculate that such an introduction has been carefully considered to gradually initiate changes in classroom assessment in Singaporean primary and secondary schools. Before the introduction of the assessment policies, there had been no nationwide targeted educational initiatives on classroom assessment, although other curricular and infrastructural policies have been introduced incrementally to de-emphasise an exclusive privileging of students’ academic achievement since the 2000s (Ministry of Education, 2005, 2013). With the recalibrating of evaluation of schools towards holistic education, there is a progressive abolishing of ranking and banding of schools by using absolute academic results (Heng, 2012). Rather, schools are encouraged to plan and evaluate their programme in a holistic manner. This involves emphasising nurturing students’ holistic growth, engagement of parents, and supporting teachers in their development and ensuring their wellbeing. Developing holistic or balanced assessment practices will seem like a logical transition to such bold moves in the Singaporean school landscape.

Methodology

The findings reported in this article form a critical part of the results of a 3-year PhD research project that investigated a group of Singaporean case-study teachers’ different experiences of thinking about and enacting classroom assessment. The use of the case study as a central methodology extends the studies of implementation of new classroom assessment practices in North America, the United Kingdom, New Zealand, and Hong Kong, which pursues a middle-ground approach between large-scale surveys (e.g., Brown, 2011; Chen, Brown, Hattie & Millward, 2012; Pedder, 2006) and individual ethnographical studies (e.g., Wong, 2007). Given the tendency of researchers in Singapore and beyond, to make use of large-scale surveys for investigating educational policy impact (e.g., Hogan et al., 2013; Koh & Luke, 2009), the descriptive case studies provide an invaluable medium to describe and relate carefully their views and practices (which are directly grounded in their contingency of classroom work) in a rigorous and holistic manner. Overall, the research has adopted a pragmatic epistemological stance of knowing teachers’ views and practices (Dewey, 1938, 1939) with a view to understanding what Sellars (2007) calls the “space of reasons”. This space is introspective. It is where teachers think and act, and they navigate this space within a practical context of tensions, constraints, and possibilities. Here, what teachers deem to be knowledge rests not so much on how strongly a fact correlates with the world, as in their personal and subjective judgements, which are mediated by their social contexts, their own past experiences, and by what those around may allow (or disallow) them to say or do. Threaded throughout the literature of research on the relationships between teachers’ views and practices is an appreciation of the depth, sensitivity, and location-specific knowledge of a teacher’s professional journey.

Participants

In the main study, six case-study teachers (see Figure 1) were selected from a pool of high-achieving Singaporean teachers who had been successfully admitted to a well-subscribed Master of Education course, offered by the only teaching college in Singapore. The entry requirements for being admitted to this master’s programme served as important criteria for selecting case-study teachers for the research. These teachers had good, if not exemplary, teaching track records and were also mostly holding leadership roles in their schools. Six teachers were finally chosen according to the criteria above and diverse backgrounds in terms of the type of schools, years of teaching, and leadership responsibilities.

Procedure

The teachers were observed for at least 12 lessons over a period of 7 months to make sure that lessons before both the examination and the non-examination season of the school terms were observed. In addition, the case-study data collection included at least 4 to 5 hours of individual interviews with each teacher (inclusive of the short pre- and post-lesson chats) based on questions derived from a phenomenographic framework (Marton & Booth, 1997) of interview and observation. At the end-of-the-lesson observation and interview sequence, the teachers were asked to complete a Q-sort of 45 statements on classroom assessment. This Q-sort was to compare the teachers’ views of classroom assessment with those of 35 Singaporean teachers who were comparable in terms of years of teaching experience, teaching subjects, leadership responsibilities and profiles of schools. The Singaporean teachers had earlier taken part in a pilot phase of Q-sorting. This article reports mainly the methodological considerations and issues regarding the use of Q-sort, which is an essential data-collection procedure in Q methodology (Q). The findings from the Q-sort shed light on how unique the views of case-study teachers were, compared to a larger group of teachers in Singaporean schools. In accordance to ethical guidelines from Ministry of Education, the participants understood and agreed to their participation without any duress, and remained informed of their rights to withdraw from the research for any or no reason. Throughout the research, their entitlement to privacy and rights to confidentiality and anonymity, were ensured.

Q

Q was developed by the English psychologist and physicist William Stephenson in the 1930s to explore the subjective viewpoints of groups of individuals (Brown, 1980, 1993, 1995). A crucial premise of Q is that subjectivity of viewpoints is communicable and can be studied systematically. When different views are expressed, read, and discussed openly, a rich resource is created for deliberation on and understanding of various collectively held viewpoints of a particular controversial topic for different individuals. Factor analysis resulting from the analysis of data represents viewpoints or clusters of subjectivity that are currently “operant” (Brown, 1980, 1993)—these are viewpoints that are currently “in action” among individuals in a community, but are not absolute in any way. Rather, the viewpoints are contingent on a specific time and place of—and purpose for—conceptualising an issue.

Participants of Q are asked to study various statements (“Q-set”) that represent many possible views of a topic, or a range of possible understandings of the topic under investigation. They are then asked to rank-order the statements according to their preferences by placing them (“Q-sort”) in a specific order, and stating their extent of agreement with their own view. By Q-sorting, participants give their subjective meanings to these statements and, in doing so, provide a way of expressing individual subjective viewpoints on how they agree or disagree with the different statements (“subjectivity”). A number of similar responses are then discovered through a statistical factoring process that characterises the viewpoint of a particular group of individuals (“intersubjectivity”). Q begins with the notion of a finite diversity of views that different individuals can express or construct about a topic (Kitzinger, 1986), and ultimately helps both them and researchers to systematically clarify and make sense of the subjectivity of viewpoints. The interest focuses on how the topic is seen from the standpoint of the individuals living it, by bringing to the individuals’ attentions all the other possibilities of conceptualisation.

When Q is positioned within a pragmatic approach, it is a mixed method which enables researchers to explore tacit decision-making processes in wide-ranging social interplay between people in practical situations in education, health-care, advertising and, lately, social networking and gaming fields. Q enables researchers to collect and study viewpoints as knowledge which is not seen as absolute in any way, but is multiple and contingent on time, place, and purpose. Crucially, Q specifically enables and expects different individuals to express their opinions in their Q-sort in a structured, participatory, and safe environment. The method employs a distinctive factor-statistical analysis to identify groups of viewpoints from participants who sort a pool of statements in comparable ways. Q is also a useful research tool to name and depict the textuality of the discourses’ interplay, and, through further interpretation, to map out their possible relations to one another (Watts & Stenner, 2012). This methodology was particularly important for the study of controversial topics (see, for example, van Eeten, 2000), providing researchers and policy-makers with different ideas for interrogating subjective opinions and suggesting opportunities for consensus-building. Brown (1995) goes so far as to argue that there is no other method that matches Q’s versatility and reach, and which comports so well with keeping up with changes in views of contemporary research topics. Q has not been well-known in education assessment research, even though it has been reported to be a useful methodology for studying different teachers’ views or perspectives (Anderson, Avery, Pederson, Smith, & Sullivan, 1997; Lim, 2010; Rimm-Kaufman, Storm, Sawyer, Pianta, & LaParo, 2006).

One of the major criticisms of Q is its atypical use of factor-statistical analysis of a small group of participants’ Q-sort. The details of how Stephenson had worked out Q by departing away from the traditional factor analysis of large-scale survey data are beyond the scope of this article, but can be read on websites such as www.qmethod.org. Suffice to say that Q recognises the ideographical nature of an individual’s view, and explores how the statistical factors, profiles of individuals, and interview data need to be considered systematically and holistically, without privileging one or the other. Q affords to researchers the wholeness of individual and group’s views espoused in Q-sorts, while comparing where the individual or group are coming from through interviews. Therefore Q enables the pursuit of empirical discoveries of quantitative and qualitative kinds (Stephenson, 1936). The promises of such a “qualiquantological” approach—or hybridisation of qualitative and quantitative methods—is what has intrigued many interdisciplinary teams of researchers (e.g. Stenner & Stainton-Rogers, 2004) who have tested its potential, despite the criticisms and challenges.

Q-sort

In total, 41 teacher participants (including the six case-study teachers), who were teaching different subjects in different schools or at the Ministry of Education, and with comparable years of teaching service, were invited for Q-sorting (see Table 1).

Table 1. Teaching profile of n = 41 teachers by type of school, years of teaching, and subject taught

Alexander’s categorisation of classroom practices (1992), and phenomenographical dimensions (Marton, Dall’Alba, & Tse, 1992; Marton & Booth, 1997), provided the basis for using 45 statements of classroom assessment practice in the Q-sorting process. These statements were representative of the variations in possible meaning of classroom assessment as experienced by Singaporean teachers, as shown by two earlier pilot studies. The pilot studies involved focus-group interviews and trialling the use of Q by 82 Singaporean teachers. The interview questions made use of a phenomenographical framework of identifying what is critical for distinguishing one way of experiencing classroom assessment from another, using minimum dimensions of variations (Marton et al., 1992). This focus on critical aspects of meaning facilitates the search for logical relationships between different meanings. A particular classroom assessment practice, according to Marton et al. (1992), can be understood by studying “what” and “how” a phenomenon is experienced, with an accompanying intention of “why” that may be articulated. Extracts of Q-statements of classroom assessment from each category (Alexander, 1992) and dimension of classroom assessment (Marton et al., 1992) are illustrated in Figure 2.

A web-based version of the Q-sort was designed between September and November 2011 to offer teachers the privacy and convenience of sorting the statements in their own time. In designing the website for a web-based Q-sort, this research referred to a number of studies that used either electronic surveys or online Q-platforms (e.g., Birnbaum, 2004; Rimm-Kaufman & Sawyer, 2004), and consulted several computer programmers. The findings of Reber, Kaufman, and Cropp (2000)—that there was no apparent difference in the reliability or validity of the manual and online methods of administration of Q-sort—were particularly reassuring.

Analysis of Q-sort

Of the 41 teachers, five (including two case-study teachers) decided to Q-sort twice based on their own reasonings of subject differences (e.g., music vs maths) or the different positions (e.g., teacher vs head of department) that the teachers held in the school. After analysis, a total of four factors resulted in each factor component containing at least three teachers’ Q-sorts. Factor loadings through principal component analysis (PCA) and varimax procedure of ±0.52 or above were significant at the p < 0.01 level and account for 55 percent of the Q-sorting variation. The statistical conditions for the isolation of factors were based on the standard Kaiser-Guttman criterion (Guttman, 1954; Kaiser, 1960) of keeping factors’ eigenvalues at 1.00 or above, and the use of Cattell’s (1966) scree test. Figure 3 displays the factor solutions, the accompanying eigenvalues and the cumulative percentage of total variation explained by the factor solutions.

A factor array shows the merging of all the Q-sorts into an ‘ideal–typical’ Q-sort (Stenner & Stainton Rogers, 2004), so that the Q-sorts from all participants who sorted similarly are represented in a single factor. Table 2 shows an extract of the 4-factor arrays, as well as the card statements. There are two types of statements that are noteworthy here: distinguishing statements, and consensual statements. The extent of agreement or disagreement is indicated by the strength of polarity (+4 suggests strongest likelihood of agreement, whereas -4 suggests strongest likelihood of disagreement). In Table 2, Statements 18, 3, 25, and 44 (“distinguishing statements”) were able to distinguish Factors 1 to 4 respectively (as given in bold type). Distinguishing statements are very important for identifying the characteristic viewpoint of the participants in the particular factor. For instance, participants in Factor 1 tended to strongly agree (+3) with Statement 18, whereas the other participants in Factors 2–4 tended to disagree with it. Many other statements ranked higher or lower for each factor by the participants (vis-a-vis other factors) were chosen to assist in the interpretation. So, for instance, Statement 3 may not be very highly ranked in Factor 2 (0), but was considered high in relation to other factors (-1, -3, -1). Consensual statements do not help to distinguish the factors. These statements did not distinguish the factors as they were consistently Q-sorted in the same way. Statement 24 is shown as such an example in Table 2. (See Appendix for the full version of the factor array for the 45 statements.)

By studying the full list of statements that were ranked relatively low or high within each factor, and also statements in each factor that were ranked higher or lower than other factors, a holistic view of how the teachers conceptualised and prioritised classroom assessment could be interpreted (Anderson et al., 1997; Lim, 2010).

Interpretations of factors

The four different Q-sort patterns represented by the four factors which emerged from the factor analysis—dilemmatic, agentic, empirical, and evolving—are summarised in the respective quadrants in Figure 4. Each factor is identified by the most relevant statements of classroom assessment that a group of teachers tended to agree with, more or less, based on their Q-sortings.

The Q-sorts of five case-study teachers could be identified in one of the factors and descriptors: Factor 1 (Dilemmatic)—Chui Seng; Factor 2 (Agentic)—Elsie, Ryan, Pei Pei (English); Factor 3 (Empirical)—Alisha; Factor 4 (Evolving)—Pei Pei (music). Pei Pei was one of the five teachers who Q-sorted twice and her Q-sortings were identified as being in Factors 2 (based on teaching English) and 4 (based on teaching music). Data from the interviews and observations of the case-study teachers were also considered for triangulation of the interpretations. The triangulations indicate that there were some contradictions between the case-study teachers’ Q-sortings and the interview findings and lesson observations. These statements were identified as problematic in terms of inconsistency with actual practices and are indicated in italics in Figure 4. All factors shared agreement over some statements—these are referred to as consensual statements previously and will be discussed separately in a later section after interpreting each of the factors.

Factor 1: Dilemmatic assessment

The 14 teachers in Factor 1—dilemmatic—broadly saw assessment as a process of understanding, making judgements, and supporting students’ learning (Statement 8). At the same time, they strongly agreed that classroom assessment would be the tests and examinations stipulated by the school or national examination board (Statements 11 and 18). They did not see the usefulness of research (Statement 39); one possible reason was that there was a considerable amount of work to prepare students for the school or national examinations. Tracey gave her reason for sorting here:

There is too much competing for our attention daily in terms of preparing our kids for the school and national examinations and other administrative duties. While useful, the research might not render benefits worth the outlay in terms of time. (Tracey, Head of English Department, autonomous secondary school, 12th year of teaching)

The time limitation was a real problem as teachers grappled with the pressing issues of preparing students for tests and examinations. There was a dilemmatic tension in what the teachers agreed about the wider possibilities of classroom assessment (Statement 8), and the pressure imposed on them by the school and the Ministry of Education to focus on school-based or national summative assessment requirements (Statement 18). This tension was clearly shown in the relatively high ranking of Statement 44. The relatively strong disagreement with Statement 45 suggested that the teachers knew in some detail what assessment could entail (e.g., formative assessment), but were probably not able to practise it in their classroom accordingly. This forced conflation of assessment with the tests and quizzes that they needed to administer in the classroom went against their better knowledge of how classroom assessment could cater to an individual’s learning. Janice and Chui Seng commented on their Q-sort:

I feel there isn’t freedom to really practise what I believe because of high-stake exams and entrenched assessment culture amongst all stakeholders. (Janice, geography teacher, government-autonomous secondary school, 6th year of teaching)

I strongly believe that there are many forms of assessment, not just those that are prescribed in the official scheme of work; and that as teachers, we should always be open to new forms of assessment. Therefore, I know that there is a clear gap in what I know or feel about assessment and what is actually happening in my classroom, as my school has set very clear and rigid guidelines about its conception of formal and informal, and formative and summative assessments. We are not allowed to deviate from the prescribed assessment formats. (Chui Seng, English teacher, independent secondary school, 21st year of teaching)

This viewpoint of classroom assessment was consistent with Chui Seng’s interview comments and observations of his lessons.

Factor 2: Agentic assessment

The teachers here recognised the need for classroom assessment to revolve around students’ classwork, homework, tests, and quizzes (Statement 3). However, they did not agree that classroom assessment should focus on the final tests and examinations alone (e.g., CA and SA) (Statement 18). This lack of agreement begs the question of what then was the purpose of emphasising the homework, tests, and quizzes that I expected to be a regular feature of their lessons. Their concurrence with statements such as Statement 17 suggests they gave priority to wanting students to “work hard” and to learn from the process of completing their homework, tests, and quizzes. The teachers here were also more willing to disregard the practicality of assessment (Statement 29) and to go that extra mile for their students to appreciate the effort of learning through giving them substantial daily class work (requiring substantial marking) and remediation classes. Lian Tin commented:

Good assessment is that continuous process of teachers’ monitoring and being in sync with the students’ learning. It is dynamic and includes giving timely, clear and constructive feedback. It may be a lot of hard work for both teachers and students but that’s what assessment should be about. (Lian Tin, university lecturer, 27th year of teaching)

The teachers agreed that they had made use of formative assessment in their lessons (Statements 20–22), which was compatible with their emphasis on class homework, tests, and quizzes (Statement 3). The observation of Elsie and Pei Pei’s classroom assessment practices suggested that this espoused use of formative assessment within frequent summative assessment testing might not always have led students to consider more closely what their learning was about, as their immediate concerns might still only be their performance in tests and examinations.

Some of the teachers seemed to openly acknowledge the limitation of using tests and examinations to support students’ learning and explained that there was more learning that could take place beyond preparation for examinations:

Examination is not everything in teaching and learning. Teachers should care about other matters pertaining to students besides just national exams. (Ai Lian, Vice-principal, aided-secondary school, 11th year of teaching)

Assessment should not help students to learn and not just for the sake of clearing national examinations. I believe assessment should tap into learners’ prior knowledge and have the necessary responses to students’ needs. Assessment should allow students room for critical thinking, problem-solving and to be builders of existing knowledge to facilitate them to be self-directed learners. (Sharon, Level Head of Science, primary school, 4th year of teaching)

The teachers here did not see research as contributing to their assessment work in the classroom (Statement 39). They also subscribed to the certainty that what they believed and knew about assessment was exactly what was carried out in their classroom (Statement 44). They did not believe that formative and summative assessment decisions in their classroom were based on what the school had already decided for them (Statements 11 and 26).

On the whole, the interpretation from their Q-sorting of the statements here was that these teachers were able to exercise their agency to make the most of classroom assessment decisions, and were not deferring to their school or to the Ministry of Education. They seemed very confident in their classroom assessment practices and did not see how research findings could advise them in their work.

Factor 3: Empirical assessment

Members of the Factor 3—empirical assessment—group were distinct from teachers in other factors owing to their significantly lesser number of years of teaching, along with the type of schools they were in—all were teaching in primary. The most distinctive feature of Factor 3 in terms of the teachers’ viewpoints was their high valuation of research and learning of best practices, as decisive references to what, how and why classroom assessment would be introduced in their classrooms (Statements 9 and 39). Gwen explained why she Q-sorted in this particular way:

Factual, statistical data is most reliable, not subjective. However, it must be complemented with shared best practices that have been tested and tried in a classroom environment, conducive to the method/practice. (Gwen, teacher, primary school, 4th year of teaching)

These teachers had a particular normative view of assessment and learning. For instance, they believed that classroom assessment should lead students to enjoy learning rather than dampening their interest in it (Statement 34). They did not agree that classroom assessment was about the tests and quizzes in the classroom (Statement 3). Rather, classroom assessment should help students learn important values and skills, which would be useful to them beyond schooling (Statement 33).

The relatively low rankings of Statements 20–23 suggest that this group did not seem to be practising formative assessment consistently in the classroom, in contrast to what teachers claimed in Factor 2. The extended lesson observations and interviews for Alisha led to an expectation that she would articulate a stronger view in the sorting about her consistent use of formative assessment practices (and the contrary for Elsie, Ryan, and Pei Pei in Factor 2). So, like the other teachers here, Alisha might actually have known more about formative assessment than other teachers in other factor groups, but she was indicating that she could only practise certain forms of it in the classroom (Statement 45). Another possible explanation is that teachers here did not feel they could exercise their full agency to practise formative assessment, owing to the school’s expectations and the imposition of tests and examinations. This points to how their Q-sorting could be a partial or even inaccurate reflection of a teacher’s circumstances and actual practice of formative assessment, in comparison with other teachers.

Factor 4: Evolving assessment

All teachers in Factor 4—evolving assessment—Q-sorted on the basis of a subject that had no formal tests or examinations in their classroom (e.g., music, character education). For the teachers in Factor 4, assessment did not exist in their classroom, either because there were no national assessment guidelines for their teaching subject (Statement 28) or there were no formal tests or examinations (Statement 18), as in the other subjects. For these teachers, classroom assessment was dependent on the teaching subjects, students, and the schools they were teaching in (Statements 41–43). Because classroom assessment is very contextual and can evolve according to different subjects, students, and schools, the practice of arranging remediation classes to better prepare students for tests and examinations (Statement 19) was too limiting. They felt strongly that classroom assessment in the form of tests and examinations could dampen the joy of learning for their subjects (Statement 34). They also considered the practical issues of introducing a rigorous large-scale assessment when there were only a few of them teaching the subject, as compared to the other subject teachers (Statement 29). In this extract, Po Meng expressed his dissatisfaction and ambivalence at the limitations of pen and paper examinations in other subjects:

Assessments tend to kill the interest in subjects, as they tend to follow exam formats and not creative means to find out how much a student has actually learnt. Yet I know it’s simple to just make use of pen and paper tests, especially as there are only a few of us teaching my subject. It’s difficult to assess character development also. (Po Meng, Head of Department of Student Leadership, independent secondary school, 13th year of service)

Overall, these teachers were thinking of ways to define and improve their assessment practices (Statement 38). Even though they were open to new ideas about assessment, they seemed to be neutral or lukewarm about the usefulness of current research on assessment practices in the classroom (Statement 39). This response may suggest that research in classroom assessment of these subjects (e.g., art, music, character/moral development and physical education) was somehow not reaching these teachers. This gap of research in classroom assessment in turn could suggest an uneven attention to research and supporting teachers of different subject areas.

Consensus statements

Consensus statements are statements in the Q-sort that do not distinguish between any of the factors. In other words, all the teachers ranked or valued these statements in roughly the same way; therefore, these statements did not help to differentiate their viewpoints of classroom assessment. The statements were not statistically significant for isolating a particular viewpoint from a group of teachers. These statements, however, were not insignificant in terms of suggesting an overall consensus in their views of classroom assessment of Singaporean teachers who participated in the Q-sorting.

Box 1 lists all the statements that were identified as having similar sorting across teachers. It can be surmised from Statements 7, 24, and 35 that the teachers were first and foremost concerned with learning as a priority in their classrooms. According to their sortings, assessment could take on many different forms and strategies in the classroom, and they believed in planning for and practising assessment that was appropriate for learners. According to the sorting of Statement 7, they also believed (albeit to a slightly more differential degree) that assessment was a feature of daily teaching in their class. (What the teaching and learning actually meant for each of these teachers needs to be examined more closely.) They also agreed that there was a need for transparency of assessment and to communicate this to students and parents. With regards to knowing a national initiative on assessment, and adhering to it, the majority of the teachers expressed that they were not guided or motivated by a particular policy, nor were they working on implementing any new assessment practice.

Box 1. Overall agreement of statements across all Singaporean teacher Q-sorters

Discussion

This Q-study has shown that there were four possible viewpoints of classroom assessment among a group of Singaporean teachers, which have been labelled as dilemmatic assessment, agentic assessment, empirical assessment and evolving assessment. The findings from the Q-study revealed that the case-study teachers’ views were not unique, and were associated with a particular group of Singaporean teachers. For each viewpoint, between four and 13 teachers were identified as similar to each of the case-study teachers. From these viewpoints, one can infer that the teachers had to conceptualise classroom assessment practices based on different competing agendas. Such a view was corroborated by Rea-Dickins (2006) and McNamara (2001), who observed that classroom assessment needed to be understood and practised in the context of competing demands and agendas. The Q-sort results also suggest that research and policy changes seem to precipitate apathy in some teachers’ professional learning about classroom assessment. This poses a tremendous challenge for school leaders, policy-makers, and teacher educators, who need to work against what Huberman (1992) described as a plateauing in teachers’ professionalism as they learn to withdraw or disengage themselves from any educational policies, research, and innovative practices, rather retreating into their own pragmatic space in their classrooms. An important implication here is that where policy-makers and school leaders are aware of the different views of classroom assessment, they need to develop and adapt policies and research findings that are responsive to specific dilemmas and tensions perceived by all teachers. This study indicates that more attention may need to be devoted to teachers who need to grapple with the pressing issues of preparing students for national examinations, and also teachers who are teaching subjects with no national examination requirements (such as art, music, and character development).

The appeal of new assessment policies, such as holistic and balanced assessment, may prompt many teachers to discover more about alternative views of classroom assessment. However, the effect of such persuasion may be short-lived as teachers may jettison alternative approaches because the cycle of reflection and action has not been fully engaged and practised. Repetitive cycles of reflection, grounded in examining their practices of classroom assessment, could stimulate teachers to ask and respond to difficult questions, which would otherwise remain tacit. The process of making the tacit explicit is not likely to happen through a single professional development session; it has to be a dialectical one in which familiar and tacit teachers’ knowing interacts with and is reshaped by newly explicit knowing.

The positive side of the Q-sort results is that there was a consensus that students’ learning was highly valued by the teachers. Such a view of classroom assessment suggests opportunities for consensus building among different groups of teachers (even among individual teachers) for coming to terms with the differences between viewpoints of classroom assessment. Many teachers made positive comments that Q had helped them to see different points of view about classroom assessment and to articulate their own views while seeing why other colleagues might or might not agree with them. The use of a research tool such as Q can double up as an important platform for professional learning within a school in a very expedient way. Q can also be used for a wide variety of teachers to compare, for instance, their views of classroom assessment, teaching, and learning across different subjects, years of service, and schools.

Although the analysis provided evidence for interpreting different Q-sorting patterns of statements about classroom assessment, it is important to note that the interpretation did not necessarily cohere with findings from the extended interviews and lesson observations of the case-study teachers. A limitation of Q is its focus on pursuing “snapshots” of reasoning through certain configurations of key Q-statements, without actually knowing the actual practices and their circumstances. It is therefore possible that an individual’s Q-sort may not be consistent with his or her practice, or it may change according to particular circumstances. For the case-study teachers, it was possible to use their actual classroom assessment practices to be more sensitive to any discrepancy of their views as espoused in their Q-sorts. The comparison of Q-sort and case-study findings suggests that teachers may overestimate or underestimate their self-assessment of their knowledge and use of formative assessment in their classes in such a one-off sorting. It may also highlight the different possibilities offered by formative assessment for different teachers at that point in time. This ambiguity suggests that much about classroom formative assessment can be covert and difficult for both participants and observer to grasp, particularly if the teacher-participants are unsure themselves (Mavrommatis, 1997). This pointed to the importance of creating appropriate professional development platforms to support and upgrade teacher’s assessment literacy, particularly regarding classroom formative assessment practices; how teachers need to know about classroom assessment practices that were aligned with a broader set of learning outcomes for the various subjects (and recently 21st-century skills and competencies), not just towards a specific set of performance goals.

Finally, I would like to mention that, even though Meilan’s data were excluded from the findings, learning more about her case in the future could bring about a greater understanding of teachers’ views and practices that are in the process of change. Thus far, this research has not been able to locate a definitive or stable set of elements that identify her within a distinct factor. In fact, her views and practices seemed to be changing, as she was being advised by her school leaders and the Ministry of Education across the span of interviews and observations. Her pattern of Q-sort could be of interest in terms of understanding and supporting teachers who are in a transitory period of changing views and practices. It is also possible that many teachers who are keen on changing their classroom assessment practices need to undergo this stage or phase of uncertainty before arriving at a more definitive set of new views and practices.

Further research could suggest a particular view and practice of classroom assessment that is sophisticated enough to blend multiple goals of teaching and learning. Where the mastery of learning-oriented approaches of teaching would align itself more naturally with formative assessment that abides more stringently to the principles of AfL (Assessment Reform Group, 2002), a more performance-oriented version of formative assessment (compare Factors 1 and 2) could be another starting point for understanding how learning and teaching can take place productively, in a different way. This is particularly so if the conceptualisation and enactment of classroom assessment is not viewed statically, but is amendable to changing dynamically according to different contexts even within the same classroom and for the same teacher.

These implications of further research work on classroom assessment in a specific Singaporean classroom context and beyond is consistent with James and Lewis’s (2012) conclusion that assessment needs to be harmonised with the understanding of learning and teaching within a prevailing social–cultural theory. For researchers of classroom assessment, this means accepting that the sociocultural dimension of the classroom matters (Carless, 2011; Brown et al., 2009); that it may be futile to expect any consistent and global theory of innovative classroom assessment practice to be practised uniformly across contexts. Rather, we have to accept and live with certain contradictions of views and see these dilemmas not as impediments of understanding, but rather as opportunities for recreating new ways of conceptualisation, particularly in terms of local sense-making. A particular teacher’s view may not be recognisable outside its intersubjective negotiated meaning, and what enables the competent, repeated reproduction of a practice, and its refinement (and recognition) while being practised—or the abandonment of an older practice—is the practical conceptualisation of what is deemed appropriate. The eventual institutionalisation of ideal practices for classroom assessment is one of the key mediators that will simultaneously shape a teacher’s view, identity, and professional life. In this study, Q has revealed a current finite diversity of Singaporean teachers’ views of classroom assessment. Such evolving understanding can help various stakeholders, such as policy-makers, professional learning developers, and researchers, begin to appreciate, clarify, and make sense of the subjectivity of enacting new classroom assessment practices.

References

Alexander, R. (1992). Policy and practice in primary education: Local initiative, national agenda. London: Routledge.

Anderson, C., Avery, P. G., Pederson, P. V., Smith, E. S., & Sullivan, J. L. (1997). Divergent perspectives on citizenship education: A Q-method study and survey of social studies teachers. American Educational Research Journal, 34, 333–364.

Assessment Reform Group. (2002). Assessment for learning: 10 principles. Retrieved from http://www.assessment-reform-group.org/publications.html

Berry, R. (2011) Assessment trends in Hong Kong: seeking to establish formative assessment in an examination culture. Assessment in Education: Principles, Policy & Practice, 18(2), 199–211.

Birnbaum, M. H. (2004). Human research and data collection via the Internet. Annual Review of Psychology, 55, 803–832.

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7–73.

Black, P., & Wiliam, D. (2005). Lessons from around the world: How policies, politics and cultures constrain and afford assessment practices. The Curriculum Journal, 16, 249–261.

Brown, S. R. (1980). Political subjectivity: Applications of Q methodology in political science. New Haven, CT: Yale University Press.

Brown, S. R. (1993). A primer on Q methodology. Operant Subjectivity, 16(3/4), 91–138.

Brown, S. R. (1995). Q methodology as the foundation for a science of subjectivity. Operant Subjectivity, 18, 1–16.

Brown, G. T. L. (2011). New Zealand prospective teacher conceptions of assessment and academic performance: Neither student nor practicing teacher. In R. Kahn, J. C. McDermott, & A. Akimjak (Eds.) Democratic access to education (pp.119–132). Los Angeles, CA: Antioch University Los Angeles, Department of Education.

Brown, G. T. L., Kennedy, K. J. , Fok, P. K., Kin, S. C. J., & Yu, W. M. (2009) Assessment for student improvement: understanding Hong Kong teachers’conceptions and practices of assessment. Assessment in Education: Principles, Policy & Practice,16 (3), 347–363.

Carless, D. (2005). Prospects for the implementation of assessment for learning. Assessment in Education: Principles, Policy & Practice, 12(1), 39–54.

Carless, D. (2011). From testing to productive student learning: Implementing formative assessment in Confucian-heritage settings. London: Routledge.

Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1(2), 245–276.

Chen, J., Brown, G. T. L., Hattie, J. A., Millward, P. (2012). Teachers’ conceptions of excellent teaching and its relationships with self-reported teaching practices. Teaching and Teacher Education, 28(7), 936–947.

Curriculum Development Council (Hong Kong). (2001). Learning to learn: The way forward in curriculum development. Hong Kong: Hong Kong SAR Government.

Dewey, J. (1938). Experience and education. USA: Kappa Delta Pi.

Dewey, J. (1939). Experience, knowledge, and value. In J. A. Boydston (Ed.), The later works (1925–1953) (Vol. 7). Carbondale: Southern Illinois University Press.

Guttman, L. (1954). Some necessary conditions for common-factor analysis. Psychometrika, 19(2), 149–161.

Heng, S. K. (2012). Keynote Address by Mr Heng Swee Kiat, Minister for Education. Presented at the Ministry of Education (MOE) Work Plan Seminar, Ngee Ann Polytechnic Convention Centre.

Hogan, D., Chan, M., Rahim, R., Kwek, D., Maung Aye, K., Loo, S. C., Yee, W. S., & Luo, W. (2013). Assessment and the logic of instructional practice in Secondary 3 English and mathematics classrooms in Singapore. Review of Education, 1(1), 57–106.

Huberman, M. (1992). Teacher development and instructional mastery. In A. Hargreaves, & M. G. Fullan (Eds), Understand teacher development (pp. 122–142). New York: Teachers College Press.

Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and psychological measurement. Retrieved from http://psycnet.apa.org/?fa=main.doiLanding&uid=1960-06772-001

Kennedy, K., Chan, J., Fok, P., & Yu, W. (2008). Forms of assessment and their potential for enhancing learning: Conceptual and cultural issues. Educational Research for Policy and Practice, 7(3), 197–207.

Kitzinger, C. (1986). Introducing and developing Q as a feminist methodology: A study of accounts of lesbianism. Feminist social psychology: Developing theory and practice, 151–172.

Koh, K., & Luke, A. (2009). Authentic and conventional assessment in Singapore schools: An empirical study of teacher assignments and student work. Assessment in Education: Principles, Policy & Practice, 16(3), 291–318.

Lim, C. (2010). Understanding Singaporean preschool teachers’ beliefs about literacy development: Four different perspectives. Teaching and Teacher Education, 26(2), 215–224.

James, M., & Lewis, J. (2012). Assessment in harmony with our understanding of learning: Problems and possibilities. In J. R. Gardner (Ed.), Assessment and learning (2^nd ed. pp. London: Sage Publications Limited.

Marton, F., Dall’Alba, G., & Tse, L. K. (1992). Solving the paradox of the Asian learner. Paper presented at the Fourth Asian Regional Congress of Cross-Cultural Psychology, Kathmandu, Nepal.

Marton, F., & Booth, S. (1997). Learning and awareness. New York: Lawrence Erlbaum.

Mavrommatis, Y. (1997). Understanding assessment in the classroom: Phases of the assessment process—the assessment episode. Assessment in Education, 4(3), 381–400.

McKinsey. (2007). How the world’s best performing school systems come out on top. Social Sector Office. Retrieved from http://www.mckinsey.com/clientservice/Social_Sector/our_practices/Education/Knowledge_Highlights/Best_performing_school.aspx

McNamara, D. (1991). Subject knowledge and its application: Problems and possibilities for teacher educators. Journal of Education for Teaching, 17(2), 113–128.

Ministry of Education. (2005). Greater support for teachers and school leaders. Singapore Government Press Release, 22 September 2005.

Ministry of Education. (2013). Towards learner-centred and balanced assessment. [Brochure]. Ministry of Education Internal Circulation Document.

OECD, Newsroom. (2013, November 3). Asian countries top OECD’s latest PISA survey on state of global education. Retrieved from http://www.oecd.org/newsroom/asian-countries-top-oecd-s-latest-pisa-survey-on-state-of-global-education.htm

Pedder, D. (2006). Organizational conditions that foster successful classroom promotion of Learning How to Learn. Research Papers in Education, 21(2), 171–200.

PERI. (2009). Report of the primary education review and implementation committee. Singapore: Ministry of Education.

Rea-Dickins, P. (2006). Currents and eddies in the discourse of assessment: a learning-focused interpretation. International Journal of Applied Linguistics, 16(2), 163-188.

Reber, B. H., Kaufman, S. E., & Cropp, F. (2000). Assessing Q-assessor: A validation study of computer-based Q sorts versus paper sorts. Operant Subjectivity, 23(4), 192–209

Rimm-Kaufman, S. E., & Sawyer, B. E. (2004). Primary-grade teachers’ self-efficacy beliefs, attitudes toward teaching, and discipline and teaching practice priorities in relation to the ‘responsive classroom’ approach. The Elementary School Journal, 104(4), 321–341.

Rimm-Kaufman, S. E., Storm, M. D., Sawyer, B. E., Pianta, R. C., & LaParo, K. M. (2006). The teacher belief Q-sort: A measure of teachers’ priorities in relation to disciplinary practices, teaching practices, and beliefs about children. Journal of School Psychology, 44(2), 141–165.

Sellars, W. (2007). The space of reasons: Selected essays of Wilfrid Sellars. In R. Brandom, & K. Scharp (Eds). Cambridge, MA: Harvard University Press.

Stobart, G. (2012). Validity in formative assessment. In J. Gardner (Ed.) Assessment and learning (2^nd ed., pp. 233–242). London: Sage.

Stenner, P., & Stainton Rogers, R. (2004). Q methodology and qualiquantology: The example of discriminating between emotions. In Z. Todd, B. Nerlich, S. McKeown, & D. D. Clarke (Eds.), Mixing methods in psychology (pp. 101–120). Hove, NY: Psychology Press.

Stephenson, W. (1936). Correlating persons instead of tests. Journal of Personality, 4(1), 17–24.

van Eeten, M. (2000). Recasting environmental controversies: A Q study of the expansion of Amsterdam Airport. In H. Addams, & J. Proops (Eds.), Social discourse and environmental policy: An application of Q methodology (pp. 41–70). Cheltenham, UK: Edward Elgar.

Watts, S., & Stenner, P. (2012). Doing Q Methodological research: Theory, method & interpretation. London: Sage.

Wong, M. W. (2007). Assessment for Learning and Teacher Development: The Experience of Three Hong Kong Teachers. Teacher Development, 11(3), 18.

World Economic Forum. (2009). The Global Competitiveness Report 2009–2010. Geneva: World Economic Forum. Retrieved from http://www3.weforum.org/docs/WEF_GlobalCompetitivenessReport_2009-10.pdf

Appendix: Factor arrays of the 4-factor solutions

The author

Before joining Nanyang Technological University–National Institute of Education (NIE), Singapore, Wei Shin Leong was head of a secondary school aesthetics department and a unit coordinator at the Curriculum Planning and Development Division, Ministry of Education (MOE). As a lecturer, researcher, and teachers’critical friend in NIE, he sees himself as an intermediary for schools, MOE, NIE and international education institutions in negotiating the ambiguities and complexities of curricular and assessment reforms, locally and internationally. He is a recent PhD graduand at University of Cambridge, where his research was on Singaporean teachers’ conceptions and practices of classroom assessment. His other research areas are assessment progressions in subject-specific domains, and expert teachers’ assessment literacy. Wei Shin also teaches undergraduate, graduate and executive leadership and MOE courses on curriculum planning and implementation, basic and advanced assessment literacy, formative assessment, and reflective practice.

Email: weishin.leong@nie.edu.sg