A comparative study of National Senior Certificate summative assessments of poetry for four South African official languages at Home Language proficiency

Abstract Over the centuries, summative assessment has entrenched itself as a powerful socio-political and educational tool for determining success, or a lack thereof, and is thus critical for shaping a learner’s life. This was particularly prevalent during the apartheid era in South Africa when racially differentiated curricula were crafted. This led to severely dysfunctional assessment practices. However, in recognition of the diversity of the nation, with its 11 official languages, and in an attempt to afford all languages equal status, at its inception in 2009, the Department of Basic Education (DBE) established a common language curriculum and guidelines for the high-stakes Grade 12 National Senior Certificate (NSC) examination. This so-called ‘equality’, however, is scrutinised in this paper, with specific reference to the assessment of poetry, a literary genre that is generally thought to be cognitively demanding. This paper investigates the similarities and dissimilarities in the Grade 12 NSC summative examination practices of four selected dominant languages studied at the Home Language (HL) level of proficiency, i.e., English HL, Afrikaans HL, isiZulu HL and Sesotho HL. Drawing on Moodley’s (2018) skills-specific knowledge for validity in the assessment of poetry, Barrett’s Taxonomy of Reading Comprehension for cognitive levels, and Umalusi’s (Council for Quality Assurance in General and Further Education and Training) criteria for determining the comparability of NSC examination papers, this paper analyses the poetry component of Paper 2 (Literature) from three sets of examinations: October/November 2019, 2020 and 2021. The paper concludes on the degree of comparability and incomparability among the four languages regarding structural and technical presentation; mark allocation and cognitive weighting; cognitive levels and scaffolding of questions; instructional verbs and degree of difficulty; validity; and the washback effect.


Introduction
The concept of assessment varies according to its purpose but is most commonly defined as an ongoing process aimed at developing learning. According to Moodley (2013: 85), assessments that comprise both summative and formative elements involve making expectations explicit and public; setting appropriate and high standards for learning; and systematically gathering, analysing and interpreting evidence for improved learning. Summative assessment is typically used to describe learners' levels of competence after the learning phase to determine grade progression. It is a powerful instrument for determining individual knowledge and skills and predicting success in different fields of study. Hence, it has become entrenched over the years across assessment bodies and across subjects globally. Indeed, despite the growing call for greater reliance on formative assessment, i.e., a construct which focuses on the actual learning process as a culture of success (Moodley 2013), summative assessment remains valuable and reliable. In South Africa, the assessment body for schooling, i.e., the Department of Basic Education (DBE), is resolute in its assessment policies across subjects articulated in the Curriculum Assessment Policy Statements (CAPS). In promoting equality in teaching, learning and assessment across the 11 official languages, i.e., Afrikaans, English, isiNdebele, Sepedi, Sesotho, SiSwati, Xitsonga, Setswana, Tshivenda, isiXhosa and isiZulu, the DBE has established a standardised language curriculum and guidelines for the high-stakes Grade 12 NSC school exit examinations.
However, the perception that some educators across the country have of inequalities in question types, instructional verbs and cognitive levels among the official home languages across the three papers, viz. Paper 1 (Language in Context), Paper 2 (Literature), and Paper 3 (Writing), cannot be ignored. Indeed, educator perceptions of inequality in assessment practices across the official languages, together with previous research on the equality and inequality of assessment practices for official languages by Moodley (2014) and Lepota et al. (2015), have been the catalysts for comparative studies of the Grade 12 NSC examination papers. Pertinent to this paper is the assessment of poetry, which is located in the study of literature, and is assessed among the 11 home languages in Section A of Paper 2 for a total of 30 marks. The focus on poetry in this paper stems from the general perception of the difficulty in teaching and learning poetry at schools; its practicality for investigation in that it is a standalone cohesive literary unit unlike, for example, the novel and drama genres; and, most importantly, the need to add to the existing literature on comparable studies of official home languages and assessment practices. Thus far, there is no evidence of such literature on the poetry section of the Grade 12 NSC examination papers for the official home languages.
The study of poetry in the language curriculum, however, remains a contentious issue. Historically, the schooling system, under the throes of the Bantu Education Act No. 47 of 1953, was designed to oppress Africans by carving their status as labourers and followers, and not thinkers and leaders. Poetry, the mathematics of language, has a higher cognitive demand. The study of poetry in the 'Bantu' language curriculum was strategically neglected, leaving in its aftermath challenges for poetry pedagogies. Engaging in poetry study can, according to Fritscher (2019), lead to the onset of metrophobia, the fear of poetry, which results in avoidance of the study of poetry. The manifestation of metrophobia is evident in the additional language curriculum. Unlike the home language curriculum, which stipulates the study of poetry, the CAPS document for First Additional Languages (FALs) taught during the Further Education and Training (FET) Phase of Grades 10 to 12 explicitly states that learners are required to choose only two of the prescribed Novel, Short Story, Drama and Poetry literary genres (DBE 2011b: 18). This marginalisation of poetry impacts negatively on learners' cognitive ability. Schools that do not opt for poetry deprive learners of the opportunity to engage with the aesthetic, linguistic and stylistic appreciation of this genre, as well as the development of substantial literacy capacities (Creely 2018: 65). The absence of poetry in the classroom is aesthetically and cognitively debilitating. Hence, Dymoke (2012: 408) asserts that it is vital to capture this increasingly endangered bird in flight. Fortunately, the study of poetry is prescriptive for the home language proficiency level. Thus, the CAPS document for Grades 10 to 12 home languages stipulates that in addition to novels and drama, poetry should be studied and assessed (DBE 2011a: 13). This paper has chosen four 'dominant' languages for its critique of the assessment of poetry. As noted by Moodley (2014: 211), usages of the adjective 'dominant' are variable and weighted with different meanings in varying contexts. This paper attributes specific characteristics for language selection. English is dominant, not because of the number of native speakers, but as it is a lingua franca in South Africa. English has international status, political and educational force, and is a growing means of earning an income and surviving. Afrikaans is dominant for its political inheritance from the apartheid era. IsiZulu and Sesotho are dominant in terms of the number of speakers, at 22.7% and 7.6%, respectively, in a population of approximately 45 million people (Statistics South Africa 2011). 'Dominance' is also attributed to the number of candidates who write the Grade 12 NSC examination papers for these four home languages -an average of 46 647 in Afrikaans, 108 483 in English, 148 231 in isiZulu, and 32 509 in Sesotho (DBE 2022).
This paper focuses on the Grade 12 NSC examination because Grade 12 is the exit point in the FET band of the schooling system. The choice of the Grade 12 NSC October/November examinations across three years, i.e., 2019, 2020 and 2021, was purposeful. Firstly, they are written by the largest cohort of candidates compared to the Grade 12 NSC May/June examination papers. These examination papers are also written by Senior Certificate (SC) candidates. Secondly, the choice of three consecutive years was motivated by the most recently written examination papers at the time of investigation of the research topic. The 2022 Grade 12 NSC May/June examination papers had not yet been written and were not publicly available from the DBE on their website. Thirdly, a longitudinal study, which would incorporate a number of previously written Grade 12 NSC examination papers, was not possible due to the short-term nature of the study and the practical length limitations of this article. Indeed, a longitudinal study would be more appropriate for postgraduate academic study. The examination questions analysed for this paper are public material and may be found on the DBE (2021b) website.
The aim of this paper is to investigate the degree of equality and inequality in the poetry questions across the four chosen home languages with regard to the offerings of the CAPS document for poetry pedagogy; alignment between the test design and the DBE's examination guidelines; alignment between the test items and the requirements of Barrett's (1968) Taxonomy of Reading Comprehension and scaffolding of questions; mark allocation and cognitive weighting; instructional verbs and degree of difficulty; validity; and the washback effect as fundamental principles of assessment.
The authors of this paper declare that they are both external moderators and are aware of the danger of unconscious bias. Thus, they have consciously worn the lens of researchers and cross-analysed the relevant examination papers.

(In)equality across languages
The majority of countries worldwide are multilingual, yet not all languages enjoy equal status. Language equality is an ideological belief rather than a practical expression. Even though the different languages might be given equal status constitutionally, they remain paper-equality languages. Almost five decades ago, Sankoff (1976: 284) argued that all natural languages are on an equal footing in terms of their capacity for human communication and that no language, by virtue of its inherent structure, bestows any general cognitive advantage on its speakers. Dovalil (2012: 27-28) avers that legal status equality means that indigenous languages can be used in government institutions. However, it is common knowledge that indigenous languages across continents lack pragmatism because of their 'shallow equality' status, i.e., although being given official status, indigenous languages are unequal in practice to languages of economy and power (Leung 2019: 7). The tension between official languages and 'equal' languages can be seen globally. For example, in Canada, French is fast becoming a minority language, despite its status in the Official Language Act (Department of Canadian Heritage 2022). While the European Union language policy acknowledges the 'equality' of the 20 official languages across its regions, it fails to acknowledge the mother tongue of citizens that falls outside the official languages (Fidrmuc and Ginsburgh 2006;Szul 2015). In addition, even though India has over 1 600 languages and dialects, only 15 were chosen as official languages, and only Hindi and English are given economic muscle (Delican 1998). Chebanne and Moumakwa (2017) and Coyne (2022), among others, highlight the tensions that exist between the languages of education, such as English and Setswana in Botswana, as well as among English, French and Portuguese elsewhere in Africa. These authors proclaim that despite being given official status for teaching and learning in these countries, Setswana, French and Portuguese remain shallow equalities, as English claims the lion's share of power. In South Africa, even though the language policy advocates the officialism of 11 languages as representative of 98% of the country's inhabitants (DoE 2003: 5), Moodley (2003: 115) states: Perhaps the status of English in comparison to all of its Cinderella sister languages is best encapsulated in the coinage: 'All South African languages are equal, but English is more equal than others' (adapted from George Orwell's 1945 version [of Animal Farm]).
Each country undoubtedly cherishes its indigenous languages but feels compelled to give way to English, the favoured language in developing digital technology and international trade. This notion is further emphasised by Legére (2022), who highlights the increasing power that English has on the African continent, despite the existence and use of indigenous languages in its various countries.
Aligned with the National Language Policy Framework (DoE 2003), the DBE enforces equality across the official languages in their assessment policy. The CAPS document and the examination guidelines are the same for each language. Since the introduction of the NSC in 2008, Umalusi has conducted several studies on curriculum and assessment standards to monitor the transition from the SC to the new NSC. Murray (2012) and Lepota et al. (2015) undertook studies to determine the degree of comparability of the assessment of home languages, revealing inequalities among the languages in the assessment of skills and cognitive weighting. Similarly, an independent study was conducted by Moodley (2014) on the comparability of visual literacy assessment for the 2009, 2010 and 2011 NSC Grade 12 October/November examination papers of six home languages, viz. English HL, Afrikaans HL, isiZulu HL, isiXhosa HL, Sepedi HL and Sesotho HL. The study found elements of both equality and inequality, raising concern that despite the equal requirements for each of the official home languages with regard to CAPS and the Grade 12 NSC Examination Guidelines, significant discrepancies exist in text choices and the skills assessed among the different languages.

Exploring poetry as a literary text
Poetry is quintessentially a stylistic portrayal making use of language manipulation to convey meaning and act as rhetoric. It has a historical, political, cultural and critical heritage and is a cornerstone in society's moral, creative and intellectual progress (Rorty 2007). The notion of what poetry is has attracted numerous acuities over the years. Famous poets and authors WH Auden (1989), ST Coleridge (1990), DH Lawrence (in Brandabur 1973), RS Thomas (2004) and N Sarraute (in O'Discroll 2008) believe that poetry is a verbal contraction; it is about the best words in the best order; it demands of its readership a new effort of attention; it is that which reaches the intellect by way of the heart; and it is what makes the invisible appear, respectively. These definitions of poetry allude to its aesthetic mystery and need for intellectual engagement. It thus bolsters the pre(mis) conceptions of the study of poetry, as documented by Hughes and Dymoke (2011: 49-51), i.e., that poetry is boring, inaccessible, a frill, too difficult to evaluate and a solitary art, it is for the elite, and that analysis is at the heart of understanding poetry. However, these perceptions are myths which can be laid to rest by culturally relevant and inclusive pedagogies.
The teaching and learning of poetry is a double-edged sword. On the one hand, it is perceived as an insurmountable activity that is meant for the elevated in thinking, and on the other hand, it is praised for its aesthetic, literary and intellectual benefits. Various studies, such as those by Benton (1999) and Pike (2000), show that many teachers shy away from teaching poetry. Benton (1999: 521), for example, indicates that teachers feel a sense of inadequacy about their own knowledge and teaching skills where poetry is concerned, and that, unfortunately, some children have rarely encountered poetry of any kind. On the other side, Pike (2000) documents the hostility and resistance learners experience toward poetry. When one approaches a poem as if it is a prose text, disappointment and frustration may follow. Pike further notes that this attitude is exacerbated when learners are expected to study poems of the great traditional poets of the pre-20th century, as they represent different cultures and linguistic challenges. Despite these perceptions of poetry, Hughes and Dymoke (2011: 51) submit that although poetry does possess a certain aura of mystery, which is part of its appeal for many, it is not something to be feared. Similarly, Pike (2000) asserts that just because poetry might be perceived as foreign within the literary genres, garnering meaning and appreciating the aesthetics conveyed by the style of the poet can be taught, and individual responses to poetry can be elicited. This is the reason Hess (2003) further provides multiple benefits of studying poetry: it encourages participation that almost no other text can produce; it fosters language learning and production through real-life experiences; and it provides a concentrated version of a parallel life.
In promoting the study of literature, the CAPS for English HL (DBE 2011a: 10) states: The main reason for reading literature in the classroom is to develop in learners sensitivity to a special use of language that is more refined, literary, figurative, symbolic and deeply meaningful… [It is] an added method of revealing, reinforcing, and highlighting their ideas.
More specifically, in encouraging the effective teaching of poetry, the CAPS (DBE 2011a: 11) distinguishes between the teaching of poems and the teaching of poetry, and states that 'Poetry should be taught, not poems'. This emphasises the need for teachers to teach poetry holistically rather than in isolation. The CAPS also lists literal meaning, figurative meaning, mood, theme and message, imagery, figures of speech, word choice (diction), tone, rhetorical devices, emotional responses, lines, words, stanzas, links, punctuation, refrain, repetition, sound devices (i.e., alliteration, consonance, assonance, rhyme, rhythm and onomatopoeia), and enjambment as aspects to be studied for an enhanced understanding of the intended message of the poet (ibid.: 23). The teaching and learning of these poetry skills not only enhances cognitive and aesthetic appreciation but prepares learners for summative assessments that include both essay and contextual questions.
A skills-based approach for assessing poetry The outcomes identified in the CAPS document in the preceding section are aligned with Khatib and Rahimi's (2012) and Moodley's (2018) discussions on the skills-based approach to teaching poetry. Moodley (ibid.) itemises the following skills for teaching and assessing poetry in the classroom: structure, sense (overall meaning), the poet's or persona's intended message, theme(s), poetic devices and figures of speech, diction, mood, tone, atmosphere, and how they are conveyed and may be used as rhetoric. Therefore, it stands to reason that for an assessment of poetry to be valid, the assessor must test poetry skills.
Assessment is integral to teaching and informs the teacher of what the learner does and does not know. According to Brown (2004), a good assessment task must adhere to key principles of assessment, viz. practicality, reliability, validity, authenticity, fairness, feedback, and washback. Moodley (2021), drawing on Brown (2004), highlights the characteristics of each of these principles. The practicality of an examination involves the environment and its suitability for the test to be administered; well-trained, ethical invigilators; and that the examination can be completed in the given time. The reliability of an assessment task refers to the consistency and dependency of the assessment; if one administers the same (or a similar) assessment task to the same (or a similar) set of learners on a different occasion, it should yield similar results. If it does not, then the examination is unreliable. To achieve reliability, the assessment design must comply with a subject-specific framework within which the specific skills are located. Validity in assessment concerns the extent to which inferences made from the assessment results are appropriate, meaningful, and useful in terms of the purpose of assessment. It thus answers the question: Does the assessment task measure what it is supposed to measure? Therefore, for an assessment task to be valid, it must be skillsspecific. The principle of authenticity addresses the relevancy of the assessment task to learners at both the local and global levels; the accuracy of information; measurability of what it is meant to assess; whether the assessment is current, pertinent, and representative of the challenges within a given discipline; whether its design reflects the complexity in content and language for the target group; and whether it stresses depth more than breadth.
Fairness in examinations signifies the avoidance of bias for gender, sexual orientation, race, culture, religion and interest and the avoidance of stereotypes and clichés. If an assessment task flouts the principle of fairness, it compromises reliability and validity, and impacts the washback effect negatively. Feedback is an essential component of the assessment process; for learning to be optimal, the assessor or teacher must provide either quantitative feedback in the form of scores against marking guidelines and rubrics and/or qualitative feedback in the form of comments. Finally, washback (also backwash) is critical for continued learning. It is a facet of consequential validity and answers the question: What is the impact of the test on learners, educators, test-designers and curriculum developers? An assessment task could have positive washback, i.e., the elements of the assessment are used to promote learning, and/or negative washback, i.e., the elements of the assessment thwart learning. It is important to note that the washback effect is not a dichotomous one; rather, it falls on a continuum of positive on the one end and negative on the other. That is to say, a single assessment design could have both positive and negative washback effects of varying degrees. These seven principles of assessment clearly denote that they are equally applicable across the examination of the different languages.
Despite the numerous benefits of teaching poetry, its standardised assessment practices are contentious. While positive washback can drive the curriculum, negative washback leads teachers to teach to summative assessments rather than the curriculum. Teaching to the curriculum provides opportunities for both assessment for learning and assessment of learning. Assessment for learning is generally formative assessment, which takes place during a learning activity, and where the priority is promoting learning among learners by engaging them in the thinking process (Moodley 2013). It differs from assessment of learning or summative assessment, where the goal is to certify competence according to a present norm. Bolton and Elmore (2016: 139) argue that poor assessment practices have a detrimental effect on learners. Pedagogically, Andrews (1991: 75) asserts that many of the disempowering approaches to teaching poetry are a result of the constraints of having to teach within the requirements of examination guidelines of an assessment system.
In addition, Lindell (2020) highlights that the security and predictability attached to examinations by educational systems dilute the authenticity of the examination process. Similarly, Cooper (2020: 16) acknowledges that the challenges to teaching poetry in a way that is meaningful and empowering for learners are a consequence of the centrality of high-stakes examinations, which are indicative of the infiltration of neoliberal ideology and values into education systems in South Africa and elsewhere. However, Moodley (2018) contends that using the skills-based approach for teaching and assessing poetry ensures adherence to the principles of validity, fairness and authenticity in testing; yields skills; develops cognition; and promotes positive washback.

Quality assurance of poetry in the Grade 12 NSC examination paper
Within the genre of literature (NSC Examination Paper 2 for all official home languages), four of the 12 poems assessed -one essay question and three contextual questions -are prescribed by the NSC Examination Guidelines. The length of the poetry essay is 250 to 300 words for disjunctive languages and 190 to 240 words for conjunctive languages. Each question is set at 10 marks. Candidates are required to choose any two of the set poems and answer questions on the compulsory unseen poem, which is a contextual question also set at 10 marks. The poetry questions have a total of 30 marks. However, for the purposes of this paper, only the prescribed poetry questions are analysed.
The CAPS for Official Languages: Home Languages are the same; however, for the purposes of this paper, the English version is used. The CAPS for Official Languages (DBE 2011a) outlines its nature and purpose and provides clarity on the depth and scope of the content to be assessed, as well as assisting teachers to prepare candidates for the examinations adequately (DBE 2021a: 3). When setting Paper 2, CAPS advocates the application of Barrett's (1968) Taxonomy of Reading Comprehension. Furthermore, the NSC Examination Guidelines indicate that in terms of cognitive demand, the various questions should be proportional according to the levels described in Barrett's Taxonomy of Reading Comprehension, viz. 40% lower-order questions (LOQs) (levels 1 and 2), 40% middle-order questions (MOQs) (level 3), and 20% higher-order questions (HOQs) (levels 4 and 5) (see Table 2). Questions can be divided into different levels of difficulty within the particular cognitive level (DBE 2021: 20). The NSC Examination Guidelines (ibid.: 31) also provide a comprehensive list of the types of questions that can be posed. This includes instructional verbs such as what, name and state for LOQs; why, explain and substantiate for MOQs; and critique, evaluate and judge for HOQs. However, a caveat is that the instructional verbs provided in the list are not always cast within a specific cognitive domain; the demands of the question as a whole -and not the instructional verb -determine the cognitive domain.
Assessment designs for each subject being examined are quality-assured by Umalusi -an independent body that applies set criteria in evaluating the quality of the examination papers. External moderators are assigned the task of ensuring that the standards are met by applying set criteria for the NSC qualification received by each candidate. The framework for quality assurance comprises a total of nine sets of criteria and 60 minimum standards (Umalusi 2021). For the purposes of analysis of Paper 2 (Poetry Section), the foregrounded criteria and minimum standards in the Table 1 are used.
Content coverage and cognitive skills are two of the various criteria applied in evaluating examination papers. In quality-assuring the poetry in examination papers for content coverage, external moderators must ensure compliance with the content of poetry teaching and learning. This includes skills such as structure, sense, message, theme(s), poetic devices, figures of speech, diction, mood, tone, atmosphere, and rhetoric. In addition, external moderators must determine an equitable correlation between the mark allocated to a question and its level of difficulty. The criterion cognitive skills is a significant yardstick for analysing the examination papers for the stipulated cognitive distribution according to Barrett's (1968) Taxonomy of Reading Comprehension. In addition, the questions set should be carefully scaffolded, beginning with LOQs, then building up to MOQs, and finally, HOQs.

Data and analysis Research design, data source and analytical framework
The authors used a qualitative approach for analysing the four questions on prescribed poetry in Paper 2 (Literature) of the October/November 2019, 2020 and 2021 Grade 12 NSC examinations for English HL, Afrikaans HL, isiZulu HL and Sesotho HL -these being the three most recent examination papers written at the time of conducting this study. The poetry questions for three of the four languages were translated into English by home language education experts and translators. The authors are fully aware of the threat of translations to meaning and that some instructional verbs, for instance, can get lost in translation because of the syntactical differences between languages. In making every effort to ensure accuracy in translations, the relevant languages underwent double translation processes for the purpose of verification. This paper thus takes cognisance of the errors caused by translation in its analysis and interpretation.
The framework of analysis emerged from the poetry-specific skills highlighted by Moodley (2018) and the quality assurance of the NSC examination papers (Table 1). Each question and sub-question for the four languages were analysed to establish their quality and to make cross-comparisons thereof using the following criteria, i.e., in alignment with Table 1 Table 2) and the scaffolding of questions. (iv) Instructional verbs and degrees of difficulty.
(v) Validity and invalidity of the examination papers. (vi) The washback effect of the examination papers.   Barrett (1968) developed the Cognitive and Affective Taxonomy of Reading Comprehension comprising five cognitive levels, viz. the literal, reorganisation, inferential comprehension, evaluation, and appreciative levels. Organised in a pyramid, the basic principle of the taxonomy is that questions should be scaffolded in the teaching and learning process. The first two levels are LOQs, the third level generates MOQs, and the final two levels yield HOQs.

The structural and technical (dis)alignment of the home languages in the assessment of poetry
The questions on the October/November 2019, 2020 and 2021 NSC Grade 12 examination papers for the four home languages are aligned with the structural requirements of the NSC Examination Guidelines. The examination papers were downloaded from the DBE (2021b) public website, and the questions analysed are from these versions. Each language has four prescribed poems, beginning with an essay question (Question 1) for 10 marks, followed by three contextual questions (Questions 2, 3 and 4) at 10 marks each. In addition, each examination included an unseen poem. However, while each poem was aligned with the requirement of a total of 10 marks, the marks allocated to the sub-questions of each of the contextual questions differed across the languages. For each contextual question in all three papers, English HL had four sub-questions allocating 2, 2, 3 and 3 marks. Afrikaans HL, on the other hand, had six sub-questions for each contextual question, allocating 1, 2 and 3 marks in no specific sequence across the three poems within a single examination paper and across the three years. IsiZulu HL set five sub-questions for each contextual poem, allocating 1, 3, 2, 2 and 2 marks for the question on each of the three poems in 2019. However, in 2020 and 2021, the mark allocation changed to 2 marks for each of the five sub-questions. Sesotho HL shows variance in the number of questions and mark allocations for each poem within a specific year and across the three years. In 2019, Questions 2 and 4 comprised five sub-questions each, and Question 3 included six sub-questions. The sub-questions in Questions 2 and 4 were allocated 2 marks each, and in Question 3, the allocation was 1, 2, 2, 2, 1 and 2 marks for each sub-question. In 2020, Questions 2, 3 and 4 comprised six sub-questions each, with Questions 2 and 4 being allocated 1, 1, 2, 2, 2, and 2 marks, but Question 3 was allocated 3, 1, 2, 1, 1 and 2 marks for the sub-questions. In 2021, Questions 2 and 3 comprised five sub-questions each and Question 4 six sub-questions. The marks allocated for each sub-question differed. Question 2 was allocated 2, 1, 2, 2 and 3 marks; Question 3 was given 2, 2, 2, 1 and 3 marks; and Question 4 was worth 1, 2, 1, 1, 2 and 3 marks.
The lack of alignment in mark allocation for each sub-question across the four chosen languages may be attributed to the construct of the NSC Examination Guidelines, which prescribes the total number of marks per question but not the number of marks for each sub-question. As the weighting of each question should be determined by its cognitive demand, i.e., fewer marks being allocated to LOQs and more for HOQs, the variance in marks within a single examination paper for Afrikaans HL and Sesotho HL suggests a lack of comparability between their choice questions. Similarly, the equal distribution of 2 marks for each sub-question for the 2020 and 2021 isiZulu HL examination papers indicates incongruence between the cognitive demand and mark weighting. English HL shows a consistent mark allocation of 2 marks for the LOQs, and 3 marks each for the MOQs and HOQs. These findings suggest that the DBE provide explicit guidelines on mark allocation for each sub-question. However, this could have ambivalent consequences. On the one hand, it will stabilise the mark scheme across the home languages, but on the other hand, fixed mark allocation patterns across the years and across home languages are mechanistic. Fixed mark allocations might impede the flexible assessment of poetry, which must take into consideration the nuances of each home language. Clearly, there is a need for the professional development of assessors and moderators on assessment techniques that include mark allocation and weighting.

Mark allocation per question and correlation with cognitive demand(s)
Two quality indicators in Umalusi's instrument for the moderation of NSC examination papers regarding cognitive skills are that there is a correlation between mark allocation and cognitive skills, and that choice questions require the same cognitive skill. The English HL examination papers across the three years demonstrate consistency between skill and the number of allocated marks.
With reference to Table 2, the LOQs are allocated 2 marks, and the MOQs and HOQs are allocated 3 marks each. Choice questions are also of the same cognitive level. As a point of illustration, in the 2019 English HL examination paper, the first LOQs for each of the three contextual poems are: In that context, (i) and (ii) are comparable for the skills assessed and their cognitive weighting. However, (iii) is slightly more difficult in its use of 'suggest' compared to 'convey' in (i) and (ii).
Questions that tested higher cognitive skills, such as diction, imagery and tone, were allocated more marks.
For Afrikaans HL LOQs, there is a consistent correlation between the cognitive demand and mark allocation across the three examination papers. However, the allocation of marks for the MOQs and Example (iv), requiring an evaluation of metaphor, is allocated 2 marks -1 mark for the identification and 1 mark for a discussion. While the former is a fair mark, the number of marks for a discussion is underweighted. In (v), a sustained metaphor is part of the question, which requires a skill of higher demand, but it is allocated only 1 mark. This question is thus also underweighted. However, in (vi), the allocation of 3 marks is fair as the cognitive demand is greater. In addition, the degree of comparability of choice questions in Afrikaans HL is compromised by the testing of different skills across the four chosen languages and within a single examination paper. For example, in 2021, the first questions for each of the three contextual Afrikaans HL poems were: What does the title of the poem refer to? (2021, Question 2.1, 1 mark) (viii) Why is the title repeated twice in the poem? (2021, Question 3.1, 1 mark) (ix) Substitute 'Dit' with four (4) words in lines 1 to 3. (2021, Question 4.1, 1 mark) The first question, (vii), requires an understanding of the title; (viii) requires knowledge of rhetoric; and (ix) requires an understanding of vocabulary in context. The remaining questions also illustrate the testing of different skills within a single examination paper. The isiZulu examination papers across the three years are consistent for their comparability of cognitive skills and mark allocation. For example, in 2019, each of the first contextual questions assesses the identification of a figure of speech and the final questions require a commentary on imagery used in the poem. Similarly, the remaining questions for each of the three poems assess the same skill. Thus, the choice questions all test the same poetry skill. However, the mark weighting across the three examination papers is questionable. For example, in 2019, 3 marks are awarded to an LOQ that requires an explanation of the given lines, but 2 marks are allocated to an HOQ: While both questions (xv) and (xvi) test a component of imagery, (xvii) assesses the reader's response to the poem. Hence, while some questions across the poems are comparable for the skills tested, others are not. The quality indicator of correlation between mark allocation and cognitive skills in the Sesotho HL examination papers also demonstrates varying degrees of consistency across the questions and across the three years. For example, in 2019, the questions on 'refrain', which are the MOQs for each of the poems, have been correctly allocated 2 marks each. However, all sub-questions for Question 2 and Question 4 have been allocated 2 marks each, suggesting a lack of differentiation between the cognitive skill tested and the mark allocation. In 2020, in Question 2, while the first two LOQs have been correctly allocated 1 mark each, the third question: 'Provide the explicit meaning of line 14', also an LOQ, was awarded 2 marks. In 2021, however, there is a clear correlation between the cognitive skill and mark weighting. For example:  Oliver et al. (2004) maintain that setting questions to cognitive levels ensures balance and learner cognitive mastery. Generally, there are two models of analysis of cognitive levels. One model attributes the number of marks to the category that carries the greater cognitive weighting. The other model, which is employed in this paper, interrogates a question for its various cognitive demands and how marks are distributed across each of the cognitive categories. For example, a question could be distributed across both the literal and inferential domains. However, the authors are mindful of Yusof and Chai's (2010) warning that not everyone can identify the cognitive level accurately, and thus manual analysis (as is the case for this paper) could result in mis-categorisation. Hence, the authors' analyses might be ambiguous for certain questions. Barrett's (1968) Taxonomy of Reading Comprehension, as dictated by CAPS for the test design of languages and used for analysis in this paper, is a pyramidal construct of five cognitive domains (see Table 2). Integral to the escalating categories of cognitive demand is the concept of 'scaffolding' for language teaching, learning and assessment. Bruner (1975) introduced the concept and related it to the support structures given to a child for cognitive development. In assessment, the structures provided are embedded in the pyramidical structure of Barrett's (1968) Taxonomy of Reading Comprehension. Test designs should begin with LOQs and are then built upon by asking MOQs, and then finally, HOQs. Despite the DBE's requirement of 40% LOQs, 40% MOQs, and 20% HOQs, this paper shows a significant degree of variance across the home languages. The analysis of the examination papers for the home languages for cognitive levels reveals disparity across the languages and within a single language across the three years. In 2019, English HL shows compliance with the requirements of cognitive distribution for each of the choice contextual poetry questions. In 2020 and 2021, however, there was an imbalance across the LOQs and MOQs of the three poems, i.e., 30% LOQs and 50% MOQs. The reason for this imbalance lies in the first question of each of the four poems: the word 'suggest' raises the cognitive level. The analysis also shows that the questions are carefully scaffolded from LOQs to MOQs and then HOQs. Generally, LOQs are signalled by the instructional verbs 'What' and 'Account for'; MOQs are signalled by 'Explain'; and HOQs by 'Critically discuss' and 'Comment on'.

Cognitive levels and scaffolding of questions
The Afrikaans HL questions show anomalous distribution within a single year and across the years of study. In 2019, Question 2 shows 60% LOQs, 30% MOQs, and 10% HOQs. Question 3 was pitched at 10%, 70% and 20% for the LOQs, MOQs and HOQs, respectively. Question 4 was set at 30%, 70% and 0% for the LOQs, MOQs and HOQs, respectively. The disparity could be a result of a combination of varying skills being tested across the three poems and varying instructional verbs that tend to signal the cognitive level. In addition, the questions are not always well-scaffolded. Question 1 begins with an LOQ and ends with two LOQs. Question 3 begins with two LOQs and ends with an LOQ. However, Question 4 begins with an LOQ and ends with an HOQ. Similarly, the 2020 and 2021 examination papers demonstrate non-compliance with the cognitive distribution requirement and inappropriate scaffolding. However, Question 4 of 2020 was well distributed.
In 2019, the isiZulu HL examination paper shows an equitable distribution across the three poems, but it was non-compliant with the required cognitive demands for the LOQs and MOQs. Each of the three questions included 60% LOQs and 20% MOQs, with the HOQs being accurately set at 20%. Similarly, in 2020, while the examination paper shows an equitable distribution across the three poems, it deviates in terms of the middle-order and higher-order cognitive demands, set at 60% and 0%, respectively. In 2021, the distribution across the three years was erratic, with only Question 3 being appropriately distributed. Nevertheless, the questions across the three years were well-scaffolded.
The Sesotho HL examination papers show a lack of compliance for cognitive distribution across the three years. In 2019, the marks were loaded in the middle-order category at 90%, 80% and 90% for Questions 2, 3 and 4, respectively. While there is some evidence of LOQs, there is no evidence of HOQs, which is not in line with the cognitive distribution stipulated by the examination guidelines, i.e., 40% LOQs, 40% MOQs, and 20% HOQs. The 2020 examination paper shows a better distribution, with the inclusion of HOQs in Questions 2 and 3, but none in Question 4. The 2021 examination paper, like the 2019 examination paper, shows a distribution of 90%, 80% and 80% MOQs for Questions 2, 3 and 4, respectively, but 0% HOQs. Scaffolding of questions was also irregular, as LOQs and MOQs were pitched differently for each set of questions. For example, Question 4 of the 2019 examination paper has the following sequence of questions: Overall, the statistics show that English HL and isiZulu HL examination papers (Poetry Section), despite lapses, are the most compliant with the cognitive distribution requirement. The Afrikaans HL and Sesotho HL examination papers (Poetry Section) show limited compliance.

Validity and invalidity of the examination papers
The integral principle of validity in testing is highlighted in Bond's (2004: 179) comment on the concept: validity is foremost on the mind of those developing measures and […] genuine scientific measurement is foremost in the minds of those who seek valid outcomes from assessment.
While there are various types of validity in testing, viz. content-related evidence, criterion-related evidence, construct-related evidence, consequential validity, and face validity (Brown 2003), the examination papers were analysed for content-related and consequential validity, geared toward answering the questions, 'Does the test item assess the skills for poetry pedagogy?' and 'What is the washback effect of the test design?' These questions do not elicit dichotomous findings but rather degrees of validity. Fox and Artemeva (2022) emphasise the need for validity in testing as they have both cognitive and social consequences.
For Question 1, the poetry essay question, the 2019 English HL examination paper focuses on imagery, punctuation and tone as they reflect the ideas in the poem. In 2020, it assessed diction, structure and tone in discussing the poet's change in attitude. In 2021, it concentrated on diction, imagery and tone in discussing a major theme in the poem critically. The common features of the three poetry essay questions were the use of linguistic and stylistic features for conveying meaning. The 2019 Afrikaans HL examination paper foregrounded figures of speech and rhetorical tools for conveying a theme. In 2020, it focused on irony for conveying meaning, and in 2021, it considered 'repetition' and structure for conveying the speaker's emotions. Like the questions for English HL, the Afrikaans HL poetry essay questions required an evaluation of the poems using linguistic and stylistic features. Likewise, the isiZulu HL 2019 poetry essay question assessed imagery and tone for conveying a theme. In 2020, the question lacked focus, with the key part of the question being, 'Explain how the poet succeeds in highlighting the meaning of the poem'. This question is overly open-ended, and the candidate could choose any feature(s) as a response. Consequentially, this might impact consistency in marking. In 2021, however, the question is specific in its foci on figures of speech, imagery, rhyme, diction and punctuation. Like the English HL and Afrikaans HL poetry essay questions, the common features of the isiZulu essay questions across the three years were the use of linguistic and stylistic features for evaluating the poems. In 2019, the Sesotho HL examination assessed how visual imagery, audial imagery and repetition are used to convey the message of the poem. In 2020, it focused on how poetic devices and word choices (diction) portray the theme of the poem. In 2021, the question was: 'Analyse this poem and show how the poet has used repetition to convey his/her message.' The focus in this poem is on the use of repetition. However, the first part of the question is rather broad, as it suggests an 'open' analysis. Perhaps repetition should have been foregrounded to obviate the ambiguity. Nevertheless, like English HL, Afrikaans HL and isiZulu HL, the Sesotho HL poetry essay questions required an evaluation of linguistic and stylistic features for conveying meaning, message and theme. It is clear that Question 1, the poetry essay questions, assessed poetry-specific skills across the four languages for the three examination papers. Thus, they can be deemed valid.
An in-depth analysis of the contextual sub-questions (in Questions 2, 3 and 4) for the home languages shows degrees of equality and inequality in the skills assessed. The 2019 English HL examination paper shows that the same set of poetry skills was assessed and sequenced equally for each of the three poems, viz. diction, stylistic device, metaphor, and appropriateness of the title of the poems. In 2020, a similar pattern was evident; the skills tested for each of the three poems were ascertaining meaning of a phrase in context, diction, metaphor and imagery, diction and/or tone. In 2021, however, the set of skills shows a degree of variance. The first sub-question for each of the three poems was about how a specific line was used to convey meaning. The sub-questions that followed were similar in that they tested diction, metaphor and diction for messages. However, while Question 4 included a sub-question on structure, Questions 2 and 3 did not. The slight variations in the foci of questions are understandable as each poem foregrounds different linguistic and stylistic features.
The 2019 Afrikaans HL examination paper shows both overlapping and different skills. For example, while the first question requires eliciting meaning, each has a different focus: (xxvii) What is the relationship between the title and the poem? (2019, Question 2.1, 1 mark) (xxviii) What serves as stimulus for the writing of this poem? (2019, Question 3.1, 1 mark) (xxix) Which word in line one refers to the title? (2019, Question 4.1, 1 mark) In addition, the remaining sub-questions for Question 2, evaluate punctuation and effect, structure, metaphor and sound effect. The remaining sub-questions of Question 3 assess diction, meaning, mood, structure and theme, and Question 4 focuses on paradox, punctuation and meaning, metaphor and diction. The focus in the skills assessment is mostly consistent, and the slight variances may be attributed to the difference in the features that are highlighted in each poem. Similar patterns are evident in the examination papers from 2020 and 2021; the skills tested are meaning in context, figures of speech, mood, tone, diction and structure. However, these are not consistent between each set of poems for each year, thus thwarting the comparability of choice questions. Nevertheless, as already indicated, different poems require different skills, and it is thus not always possible to focus on exactly the same set of skills for each examination paper.
The 2019 isiZulu HL examination paper shows that the choice questions are comparable in that mostly the same skills are tested for the three questions, viz. meaning in context, figures of speech, a foregrounded stylistic feature on each poem, rhyme and punctuation. The single invalid question for 2019 is Question 3.1, which requires the part of speech of an underlined word in the poem. Such a question is concerned with a grammatical skill and should have rather been phrased to include its impact on the poem, so as to adhere to the principle of validity. The 2020 examination paper focuses on how language is used for making meaning; questions on diction, theme, rhyme and tone appear consistently. This ensures comparability of choice questions. Similarly, in 2021, the foci of questioning in each of the three poems are figures of speech, imagery, tone and making connections in meaning. In addition, in questions on foregrounded rhetorical devices, the three different features assessed are the use of consonants, punctuation and enjambment. Unlike the 2019 examination paper, there were no invalid questions.
The 2019 Sesotho HL examination paper shows some overlaps and differences in the skills assessed in terms of the comparability of choice questions. For example, while Question 2 has one sub-question on repetition, Question 3 has two sub-questions set on the concept, and Question 4 does not have any such sub-question. In addition, while Questions 2 and 4 include sub-questions that allude to a discussion of diction, Question 3 does not. There are further inconsistencies in skills assessed regarding reader response to the text; this is assessed in Question 2 and 4, but in Question 3, the writer's viewpoint is assessed. Despite the difference in skills tested between each poem, each skill is valid. Overall, the skills assessed are repetition, diction, structure, tone and stylistic devices, viz. repetition, rhetorical question and apostrophe. The 2020 examination paper is consistent in the skills assessed; each of the three poems evaluates a rhetorical device, imagery, themes, poetic devices and the effect of punctuation use. However, the poetry-specific skill in Question 2.3, i.e., 'Provide the explicit meaning of line 14', is unclear. The invalidity of this question could have been avoided if reference was made to a specific poetic feature. The 2021 examination paper was equally valid in its assessment of poetic devices, rhetorical devices, punctuation use for effect, main ideas and reader response. However, the skills for each of the three poems are not comparable, as there is evidence of both overlapping and different skills. For example, three sub-questions in Question 4 focused on aesthetic appreciation, while Questions 2 and 3 did not. Also, while Questions 2 and 3 questioned reader response, Question 4 did not.
Overall, the poetry questions for the four home languages may be evaluated as being valid to a high degree. All questions for English HL and Afrikaans HL across the three years demonstrate absolute validity. However, isiZulu HL and Sesotho HL both show a single invalid question and may thus be evaluated as having a very high degree of validity. Despite the validity of the questions, the degree of comparability regarding the levels of difficulty of the questions is unequal across the four chosen languages.

Instructional verbs and degrees of difficulty
Instructional verbs are critical for indicating to the candidate the expectations of a question. Biggs and Tang (2007) aver that it is good praxis to align the learning outcomes with the assessment tasks. They suggest that this alignment is accomplished when the learning outcomes and the assessment tasks use the same instructional verbs that candidates are required to enact. In assessment, the verbs tend to be located within a specific domain (Munzenmaier and Rubin 2013). However, as indicated earlier, the application of a specific instructional verb from the list provided by the DBE does not necessarily inform the cognitive level; rather, it is defined by the demands of the question formulation.
In conducting the analysis on the use of instructional verbs and question formulation, the authors are mindful that direct translations of instructional verbs from one language to another could give rise to differences in nuances, creating subliminal errors. However, for the purposes of the analysis, English translations of the Poetry Section Questions 1 to 4 from the Afrikaans HL, isiZulu HL and Sesotho HL examination papers have been used. The authors acknowledge that the original questions in the relevant HL together with the translations would be ideal; however, the length of this paper is limited.
The analysis shows variability in the use of instructional verbs across the four chosen home languages. The 2019, 2020 and 2021 English HL examination papers demonstrate preciseness and clarity in the question design. The following questions across cognitive levels serve as points of illustration: Each of these examples clearly articulates what the candidate is expected to answer using the terms 'What?', 'Explain', and 'Critically discuss', respectively. In addition, each example relates to a specific aspect of the poem. The focus is on 'the feelings of the people', the reader's 'response to the frost', and 'the speaker's message about the power of the storm', respectively. There is no ambiguity regarding the requirements of the questions that assess poetry-specific skills here. With regard to the Afrikaans HL examination papers, the most common instructional verbs in the 2019 examination paper are those that require explicit responses; for example, 'What', 'To whom', 'Name', 'Which', and 'In which'. While these instructional verbs dominate the questions, other questions require stating opinions with motivation and making evaluations about an aspect of the poem being tested. A similar pattern emerges for the 2020 and 2021 examination papers. Of interest are questions that require a discussion of images, figures of speech and poetic devices. The questions are double-barrelled, complicated and lacking in focus. For example: (xxxii) Do you think the repetition of words in line 34 and 35 and in lines 42 to 46, respectively, has anything to do with the emotional state of the speaker? Motivate your answers in both cases. Also, quote a contradiction of this emotional state from lines 15 to 18. (2019, Question 3.5, 3 marks, HOQ). (xxxiv) Is the use of the words 'bros' and 'brose' with regards to the AIDS sufferer effective?
Motivate your answer in each case and indicate whether the synecdoche in line 7 links with the alliteration. (2020, Question 3.3, 3 marks, HOQ) (xxxv) Which two messages does the speaker convey in lines 11 to 15 to prove the general impressions about poets being wrong? Does the speaker allocate responsibility to poets? (2021, Question 3.5, 3 marks, HOQ) In (xxxiii) and (xxxiv), each question has three parts, and the second question in (xxxv) fails to elicit a discussion. It is assumed that a discussion is required, but if the candidate simply stated 'Yes' or 'No', it would not be incorrect. This has further complications for the marking guidelines. The Examination Guidelines specifically state that no mark should be awarded for simply stating 'Yes' or 'No'. The above questions could have been asked more succinctly and with greater focus. The isiZulu HL examination papers use a variety of instructional verbs and do so consistently across the choice contextual questions. For example, in 2019, the instructional verbs are: 'Write', 'Briefly explain', 'What', 'Explain', and 'Comment'. In 2020, the instructional verbs are: 'Give an example', 'Summarise the meaning', 'Explain', 'What', and 'Analyse'. However, in 2021, these instructional verbs, viz. 'Write (a line)', '(Briefly) explain', 'What', 'Explain', 'Comment', 'Quote', 'Examine', and 'Name', are not consistent across the choice contextual questions. The most common instructional verb in the 2021 examination paper is: 'briefly explain'. While the majority of the instructional verbs across the three years provide clear foci, the instructional verbs 'Summarise (the meaning)' and 'Analyse' in context are unclear about what is expected of the candidate. In addition, the instructional verbs, 'What', 'Name', and 'Quote' suggest a low degree of difficulty of the questions.
The Sesotho HL examination papers show a restricted use of instructional verbs. For example, in 2019, four of the instructional verbs in Questions 2 and 3 are: 'What', and the other instructional verbs are: 'Indicate', 'Paraphrase', and 'How'. Question 4 uses 'What' twice, 'How' twice, and 'Summarise' once. Similarly, in 2020, the instructional verb 'What' permeates the questions, and the other instructional verbs include 'Identify', 'Paraphrase', and 'Explain'. In 2021, there is a greater variety in the choice of instructional verbs. However, these were not consistent across the choice contextual questions. These include: 'Name', 'Briefly explain', 'Briefly define', 'How', 'What', 'Discuss', Explain', 'Comment', 'Quote', and 'Examine'. Instructional verbs such as 'Summarise' and 'Paraphrase' raise concern regarding the poetry-specific skills being evaluated. However, unlike the 2019 and 2020 examination papers, the 2021 examination paper included instructional verbs such as 'Discuss' and 'Comment', suggesting that the questions are of greater cognitive demand.
The instructional verbs used are not only indicative of the cognitive level of the question but its degree of difficulty as well. Reeves (2012: 69-70) explains that in determining the degree of difficulty of a question, evaluators must consider the intrinsic difficulty of the particular question in terms of the properties or qualities that make questions "easy", "moderately challenging" or "difficult" for the "average" Grade 12 candidate to answer. Such properties are 'content (subject/concept) difficulty; stimulus (question) difficulty; task (process) difficulty; and expected response difficulty' (Reeves 2012: 73). In analysing the questions for their degrees of difficulty in Question 1, the poetry essay questions across the four home languages and across the three years of study appropriately indicate a range of easy, moderately challenging and difficult demands. However, there exist distinct differences among the contextual questions across the four chosen languages. The English HL contextual poetry questions revealed an appropriate distribution of difficulty levels over the three years. The Afrikaans HL examination papers show a degree of compliance -the numerous lower-order instructional verbs, on the one hand, suggest that there is a low degree of difficulty in the questions. On the other hand, the complex questions show a high degree of difficulty. Similarly, the isiZulu examination papers show a degree of compliance in terms of difficulty; some questions are easy, while others are moderately challenging. Even though the instructional verb 'comment' anticipates a difficult demand, the actual questions are moderately challenging. The Sesotho HL examination papers are inundated with lower-order instructional verbs, suggesting that across the three years, the questions, except for a few, are rather easy, compromising an equitable distribution of degrees of difficulty.

The washback effect of the examination papers
Various experts in the field of washback, such as Alderson and Wall (1993), Bailey (1996) and Wiseman (1961), concur that in its simplest form, washback is the impact of a test design on candidates, teachers, examiners, curriculum designers and the like. As stated earlier, generally, washback is thought to have positive effects to the extent that it can promote teaching and learning, or it could have negative effects, such as constraining the curriculum and impeding educational goals. However, as Alderson and Wall (1993: 117) and Moodley (2015) caution, rather than being a dichotomous phenomenon, washback has impact. The same test design could have positive and negative impacts. This is evident in the analysis of the poetry questions across the four home languages. In terms of positive washback, teachers and learners across the four home languages would be conscious of the skills that are tested; these would thus be reinforced in the classroom. In addition, English HL and isiZulu HL teachers and learners would benefit from the questioning technique. However, the questioning techniques for Afrikaans HL and Sesotho HL are more likely to have a negative influence. The Sesotho HL examination papers, particularly, would have a stronger negative impact as the questions are mostly of a lower cognitive level and low level of difficulty, thwarting critical engagement with the poems. They are thus a foil for cognitive enhancement among learners.

Conclusion
The analysis of the section on poetry in the October/November 2019, 2020 and 2021 Grade 12 NSC examination papers for English HL, Afrikaans HL, isiZulu HL and Sesotho HL (Paper 2) shows inequality across the four home languages. Although these home languages respond to the same CAPS and examination guideline documents, the interpretations thereof clearly differ. Firstly, while the layout and structure of the examination papers are the same, the number of marks allocated to each sub-question differs drastically across the languages. In addition, while English HL and isiZulu HL show consistency in mark allocation across the questions for each examination paper, Afrikaans HL and Sesotho HL do not. The inconsistencies could be attributed to the Examination Guidelines, which do not provide the total number of marks for the sub-questions. The mark allocation and correlation with the cognitive demand of the questions also differ across the home languages. While English HL allocates two marks for the LOQs and three marks to each of the MOQs and HOQs, Afrikaans HL, isiZulu HL and Sesotho HL award one mark for the LOQs and either two or three marks for all three cognitive levels. The cognitive levels show disparity among the four home languages. While English HL shows a disparity between the lower and middle cognitive domains in 2020 and 2021, Afrikaans HL, isiZulu HL and Sesotho HL demonstrate a lack of full compliance with the prescribed distribution. In 2020 and 2021, English HL deviated by an overweight of 10% for MOQs. The distribution in the cognitive levels of Afrikaans HL, isiZulu HL and Sesotho HL shows considerable variance, with very few questions located in the higher-order category. Furthermore, all four home language questions test what they are meant to test, except one invalid question each in an isiZulu HL and Sesotho HL examination paper. Nevertheless, the varying instructional verbs and the question complexity indicate that different difficulty levels exist among the four home languages. English HL consists of a distribution of levels of difficulty, including easy, moderately challenging, and difficult. While the Afrikaans HL and isiZulu HL questions consist of both easy and challenging questions, the tendency is toward easy questions. Sesotho HL, however, shows that the large majority of questions are easy or of a low level of difficulty. Consequentially, the washback effect varies across the papers, except for the skills being tested. When compared to CAPS and the Examination Guidelines, English HL is most likely to promote mostly positive washback, with a few lapses, for the various criteria discussed. Afrikaans HL, isiZulu HL and Sesotho HL promote positive washback for the skills tested but negative washback for the cognitive levels, instructional verbs and levels of difficulty. Overall, the summative assessment of poetry for English HL, Afrikaans HL, isiZulu HL and Sesotho HL across the October/November 2019, 2020 and 2021 Grade 12 NSC examination papers is unequal. This raises concern over the development of the official languages towards 'equal' status. It also raises concerns about the use of marks for post-school admissions. Ironically, those languages that set easier questions escalate the possibility of admission to higher institutions. Those languages that are deemed fair, valid and reliable pose a greater challenge for student admission.
In addition to addressing its aim of determining the degree of (in)comparability across the languages, this paper should evoke debate on the issues raised. Also, it should not be seen as an attempt to promote the approaches of certain language papers and negate the practices of others. Although the poetry questions of only four home languages are analysed in this article, the other languages will, hopefully, benefit from a better understanding of the practices and approaches applied here. The authors recommend that Provincial Education Departments (PEDs) and their districts provide the much-needed professional development workshops on poetry-specific skills for teaching and learning for educators across the official languages. In addition, the DBE should respond to the urgent need for professional development workshops on summative assessment techniques -especially those on the validity of questions, mark allocation, cognitive levels, difficulty levels, and scaffolding -for the examining panels across the official languages to achieve a common understanding of summative assessment principles and practices. Furthermore, it is recommended that Umalusi engages in similar workshops engaging external moderators of the official languages on subject-specific skills and summative assessment. Ideally, however, both the DBE and Umalusi need to be in unison regarding the roles and obligations of assessors and moderators in the NSC high-stakes Grade 12 summative assessment practices. Moreover, due to the limitations of this paper, i.e., the constraints of presenting each question in its original language followed by translations, and providing the full analyses as appendices, the authors suggest the need for large-scale research on summative assessment practices of the Grade 12 NSC examination papers across the official languages. Finally, as there exists a significant gap in the research on the equality of summative assessment practices across the official languages, the DBE, Umalusi and universities in South Africa should unite in partnership to engage in research projects to address the shortcomings.