Comments from the Chalkface Margins: Teachers’ Experiences with a Language Standard, Canadian Language Benchmarks

193-219


Comments from the Chalkface Margins: Teachers' Experiences with a Language Standard, Canadian Language Benchmarks
This study examines teachers' responses to the requirement to use the Canadian Language Benchmarks (CLB) in daily planning, teaching, and assessment in governmentfunded programs for adults -Language Instruction for Newcomers to Canada (LINC) and English as a second language (ESL) in Ontario.Data for this article were drawn from a larger mixed-methods exploratory project on the mandatory implementation of portfoliobased language assessment (PBLA), which had not only changed approaches to assessment in LINC programs but enacted tools for ensuring that teachers plan and teach according to the CLB.
While the title of this article was prompted by the relatively marginal space that the CLB occupied within the larger research project examining PBLA implementation, it also reflects the focus on the experiences of the classroom practitioners, hence the modifier chalkface, defined as "the work or art of teaching in a school, esp[ecially] classroom teaching as distinct from organizational responsibilities" (chalkface, n.d.).LINC practitioners' voices continue to be marginalized by the domination of aspirational and managerial discourses.As detailed further in the literature review, these discourses have been prominent in policy making, professional LINC/ESL literature, and teacher professional development (PD) dedicated to PBLA.While academic research has included LINC practitioner perspectives, they have had a limited impact on shaping PBLA implementation, of which practical application of the CLB is the key component.Through a phenomenological lens (Usher & Jackson, 2014;Vagle, 2018;van Manen, 2014), this study is foregrounding the lived experiences of practitioners with the CLB.

Introducing the Language Standard
In 1992, LINC was established as a federally-funded national program for newcomers to Canada (Derwing, 2017;Guo, 2013).With the goal of providing a common framework of reference for this national program, the CLB document was developed and later revised (Centre for Canadian Language Benckmarks [CCLB], 2012; Citizenship and Immigration Canada [CIC], 1996;Pawlikowska-Smith, 2000).The CLB is a competencybased scale of language proficiency in English in the four skills (listening, speaking, reading, and writing) across 12 benchmarks grouped in three stages (basic, intermediate, and advanced).While the CLB has been a key document for LINC, it offers potential for use in other contexts (ElAtia, 2017).
In the absence of a national LINC curriculum, the CLB use extended beyond setting levels of English proficiency and into the realm of a curriculum, describing teaching content and methodology.While the CLB was introduced as "a framework of reference for learning, teaching, programming and assessing adult ESL in Canada" (CCLB, 2012, p. v), questions were raised about distinctions between the roles of a framework and a standard (Haque & Valeo, 2017;Smit & Turcot, 2010).The CLB declared that it was not "a description of the discrete elements of knowledge and skills that underlie communicative competence, … [not] a curriculum, [not] tied to any specific instructional method, [not] an assessment" (CCLB, 2012, p. v).However, a footnote on the same page seemed to favour task-oriented teaching: "instructional practices should focus on preparing learners to carry out contextualized 'real world' communicative tasks consistent with the CLB" (p.v).Observing such contradictions in the earlier version of the CLB document, Fleming (2010) called the disclaimer about the CLB not being a curriculum "rather disingenuous" since the "externally imposed assessment tools such as the CLB are in fact meant to control the content and methods of instruction" (p.593).Such control of the content and methods of instruction became operational with the introduction of PBLA, as described further below.
Scholarly databases produced limited numbers of publications mentioning the CLB in the context of language teaching and learning.On November 15, 2019, for Canadian Language Benchmark* as a search phrase, ERIC and Education Source each yielded 14 results, with nine items in common, and five items unique to each source.Of the 19 peerreviewed items, none examined teacher experiences.Seven articles discussed the CLB use for assessment, without explicit connections to language teaching and learning (Bruni & Irwin, 2007;Epp & Stawychny, 2001;Hudson, 2005;Norton & Stewart, 1997, 1999;Rossiter & Pawlikowska-Smith, 1999;Stewart et al., 2001).Two items represented policyarticulated vision, either of the CLB (Pettis, 2007), or PBLA implementation (Holmes, 2015), without offering empirical evidence.
Remaining publications connected the CLB to language teaching and learning to varying degrees.Among other observations, Fleming (2010Fleming ( , 2015)), pointed out the limited understanding of citizenship in the CLB.Similarly, analyzing a wider range of policy documents, Burkholder and Filion (2014) problematized linking citizenship rights to language ability as captured by the CLB 4, required for Canadian citizenship application.
Two articles (Apedaile & Whitelaw, 2012;Campbell et al., 2014) reported on designing the CLB-referenced curricula and teaching materials by dedicated teams in response to the needs of communities or programs.While Apedaile and Whitelaw reported on the design and teacher experiences with a "culturally integrated approach to teaching English" (p.127), Campbell et al. focused on the task-based feature of the curriculum.Similarly, concerns with task-based instruction guided analysis of a task from a CLBaligned LINC Home Study program (Lenchuk, 2014).Unexpectedly, the governmentsponsored resource demonstrated theoretically and methodologically outdated features contradicting the CLB.As these articles suggested, application of the language standard to curriculum resources required dedicated professional teams, which still did not guarantee successful outcomes.
Possible reasons for practitioners' challenges with utilizing the CLB were offered in Cray's (2003) review of the CLB-supporting Guide to Implementation (Holmes, et al., 2001).Cray observed contradictions and inconsistencies in the Guide.While not focusing on the CLB per se, the author concluded that it was "not surprising that teachers have not been immediately clear about what benchmarks mean to them" (Cray, 2003, p. 621).The scarcity of research on practitioner experiences with the national language standard has not been rectified.
Three additional studies addressing teacher response to the CLB were discovered through literature searches beyond scholarly databases (Haque & Valeo, 2017;Koreen, 2005;Smit & Turcot, 2010).These studies, built on data collected prior to PBLA implementation, reported the positive impact of the CLB and made recommendations for further improvements.In a master's thesis, Koreen interviewed eight teachers in Manitoba, where concerted efforts on the CLB promotion were made since 1996 (Pettis, 2015).Teacher participants reflected on changes in classroom practices, with the adoption of the task-based approach being the most significant shift.The author deliberated on the possible conflict of interest, power differences, participant self-selection bias, and the resulting questionable generalizability of the findings to a wider population of LINC teachers in Manitoba.She concluded that reported changes in teaching practices could not be attributed solely to the CLB, but that professional maturation was also responsible.
In a report funded by CIC, Smit & Turcot (2010) described a national consultation, which assisted in the revisions of the CLB in 2012.Recommendations in the report included the need for "developing and sharing quality resources," while "taking into account recognition of intellectual capital and the need to cover development costs" (p.vi).Haque and Valeo (2017) offered "teachers' own thoughts about how the CLB informs their classroom practice" (p.57), and concluded that "most instructors still want further support, including both pre-service and in-service training, mentoring, and specific resources to help guide them in the use of the CLB to inform their teaching and assessment" (p.69).To my knowledge, no other empirical study of teacher experiences with the CLB is available preor post-PBLA.
Even though the CLB document was revised in 2012 to improve "comprehensibility, clarity, consistency and relevance," and ensure "a common understanding of the CLB among the majority of users" (CCLB, 2012, p. ii), the achievement of these goals needs to be documented through the perspectives of LINC teachers, "who ultimately give the CLB the greatest purpose" (Haque & Valeo, 2017, p. 69).Therefore, this study was guided by the following research questions (RQs): 1. How do LINC teachers describe their experiences with the CLB as part of the PBLA implementation process? 2. How do LINC teachers see the relationships between the CLB and PBLA? 3. How do the relationships between the CLB and PBLA moderate teacher response to the CLB? Literature Review: From the CLB to PBLA Connections between the CLB and PBLA were aptly captured by Derwing's (2017) interpretation of PBLA as "an offshoot of the CLBs" (p.89).However, PBLA implementation was a watershed moment in the CLB transition from a document facilitating LINC students' placement and program delivery to a mandatory alignment tool for teacher planning, instruction, and assessment.Through a government-funded nonprofit, the Centre for Canadian Language Benchmarks (CCLB), mechanisms were introduced to oversee both the CLB and PBLA implementation by LINC teachers and programs.
For LINC students, PBLA protocol requires 32-40 completed portfolio artefacts for progressing to the next benchmark (CCLB, 2019a).These artefacts, provided by the teacher, ensure alignment with the CLB, i.e., being benchmark-appropriate and covering "a broad range of CLB competencies from the five competency areas (Interacting with Others, Giving/Comprehending Instructions, Reproducing Information, Getting Things Done, and Sharing/Comprehending Information)" (CCLB, 2019a, p. 27).This requirement for ongoing CLB-alignment in teaching and assessment started with mandatory teacher training, delivered by PBLA Lead Teachers, proceeded to gradual PBLA implementation across the country (Holmes, 2015), and culminated in PBLA Practice Review Framework (CCLB, 2019b).Even though the Framework claimed to be a self-assessment and monitoring tool, empirical studies of PBLA documented practitioners' discomfort and fear of the use of learner portfolios for teacher evaluation (Fox, 2014;Desyatova, 2018Desyatova, , 2020)).Such use was particularly alarming considering challenges and controversies described in PBLA literature, summarized below.
Concerns around PBLA implementation ranged from challenges specific to particular CLB levels, to issues experienced by practitioners across the country, as well as the lack of clarity about PBLA goals and purposes (Vanderveen, 2018).Fox and Fraser (2012), Abbott (2019), and Karasova (2019) concluded that PBLA protocol was inappropriate for beginners due to its heavy emphasis on metacognition, which can not be clearly communicated at the beginning stages of language learning.As for PBLA assessment function, Fox (2014) observed that the formative potential of PBLA portfolio was being thwarted by its summative evaluation use, and Mohammadian (2016) questioned the validity, reliability, and practicality of PBLA assessment.Researchers repeatedly documented taxing demands of PBLA on teacher time outside of classroom hours (Fox & Fraser, 2012;Mohammadian, 2016;Ripley, 2018), a challenge that should not be underestimated, considering precarious employment and hourly-wage-based contractual nature of compensation in the female-dominated profession (Breshears, 2019;Faez & Valeo, 2012;Haque et al., 2007;Valeo & Faez, 2013).PBLA implementation demonstrated features of a top-down reform, insensitive to practitioners' needs and challenges (Desyatova, 2018(Desyatova, , 2020)).Due to numerous controversies over PBLA and its implementation, teacher experiences with the CLB pre-and post-PBLA may differ significantly.
PBLA-related research mentions the CLB in at least one of the following three functions: 1) a level of language proficiency, an attribute to describe either learners or teachers; 2) an assessment improvement tool guiding artefacts creation in PBLA; and 3) a tool facilitating teaching and learning.The first use of the CLB as an attribute of learners and teachers does not require empirical validation; however, with the other two uses -the CLB as an assessment improvement tool, or as a document facilitating teaching and learning -a clear distinction needs to be made between the policy-articulated aspirational claims, and empirical evidence documenting the achievements in practice.The former can offer a direction for research, but should not be mistaken as an acceptable substitute for the latter.
Most PBLA-related studies acknowledge the CLB and PBLA potential to improve assessment, teaching, and learning, while also offering recommendations for increasing the potential.Fox and Fraser (2012), Fox (2014), and Ripley (2012Ripley ( , 2018) ) reported rising awareness and use of the CLB and TBLT.However, questions were raised about assessment practices (Fox, 2014;Karasova, 2019;Mohammadian, 2016;Ripley, 2018).Whereas the assessment concerns may be related more to PBLA than the CLB, the role of the CLB in teaching and learning in LINC remains unexplored.
This study, through the lens of teacher experiences, rather than policy expectations, examines the place of the CLB within PBLA as the mandatory assessment protocol in language learning programs for adult newcomers to Canada.The researcher's positionality (Rowe, 2014) has been embodied in a multifaceted identity combining academic and professional experiences of a graduate student, language teacher, teacher educator, and teacher mentor with fourteen years of experience in the LINC context.This combination of experiences drove the conceptualization and execution of this study as part of a larger project (Desyatova, 2018(Desyatova, , 2020)).To minimize the potential for bias resulting from the researcher's embeddedness in the context, established research ethics standards were followed at all stages.The author's close connection to the LINC context sensitized her to emic (insider) perspectives of LINC practitioners, rarely represented in the CLB literature.

Methodology
Data for this article -survey responses and interview excerpts referring to the CLB -were extracted from a mixed-methods dataset, collected for a larger exploratory research project on PBLA implementation (Desyatova, 2018(Desyatova, , 2020)).The project examined the experiences of various stakeholders, of which LINC teachers became the focal group in this study (N=247).

Participants
Altogether, 247 LINC teachers from across Canada responded to an online survey.The participants represented Canadian provinces in proportions commensurate with general population size, immigration patterns, and distribution of LINC programs across the country (CIC, 2010).The sample size (N=247) approached 10% of the total number of 2,500 teachers implementing PBLA/CLB in Canada (Holmes, 2015), which offers statistical support for the generalizability of the findings (Gravetter & Wallnau, 2013).Demographically, the teacher participant group was comparable to the statistical portrait of LINC teachers (CIC, 2010) in gender and age, another indication of representativeness of the sample.The interview segment of the CLB-focused dataset included excerpts from 29 teacher interviews, which allowed for sufficient minimal sample size to ensure data saturation and productive grounded theory analysis (Onwuegbuzie & Leech, 2007).

Instruments
Surveys and semi-structured interview guides were developed based on the prominent themes in the literature.Prior to the study, the instruments were piloted with two expert LINC teachers, whose suggestions on improving the clarity of Likert items were incorporated.Since increased CLB use has been a core element of PBLA implementation (Fox & Fraser, 2012;Ripley, 2012Ripley, , 2018)), four Likert-type survey items explicitly referred to the CLB (Table 3).Two of these survey items probed the CLB uptake by LINC students through the eyes of their teachers.An assumption was made that students' uptake of the CLB is part of teachers' experiences with the standard.Open-ended comments were invited after every Likert item, to provide opportunities to clarify participant-intended meanings.Semi-structured interview questions (Appendix A) did not explicitly mention the CLB, but the majority of participants (29 of 41) referred to the standard in the interviews, which became both the impetus and the qualitative dataset for this study, together with CLBrelated survey comments.

Data Collection Procedure
Invitations to participate in the study were distributed by professional TESL associations across Canada in 2017-2018.Whereas 247 LINC/ESL teachers responded to the quantitative segment of the survey described above, 69 participants offered qualitative survey comments on the CLB: 13 teachers mentioned the CLB without elaborating on an issue, 27 teachers offered a limited elaboration, such as description of a CLB-related experience, and 29 participants provided more extensive elaborations, which resulted in a total of 126 CLB-related comments.
41 teachers expressed interest in participating in subsequent interviews either in person, via Skype, or phone.Conducted by the researcher, the interviews were audiorecorded, transcribed, and re-read.During the conversations, the researcher took on the role of a listener and facilitator, following the interview guide flexibly, according to the flow of arising themes.The researchers' experience as a LINC teacher, together with an assurance of confidentiality, helped to minimize potential power differences.

Data Analysis Process
Data were analyzed both quantitatively and qualitatively using a pragmatic approach (Morgan, 2007).While the first two RQs examined lived experiences as reported by participants, RQ3 (i.e., PBLA moderating teachers' experiences with the CLB) required an integration of the findings on the first two questions with the results of primary analyses.
Neither qualitative nor quantitative data alone were sufficient to adequately answer the RQs.Limited quantitative data segment of four Likert items was complemented with open-ended survey comments (N=69, n=126; hereafter, N represents the number of participants and n the number of comments) and interview excerpts (N=29).Thematic analysis of teacher comments was guided by the grounded theory (Morse et al., 2009), from open to axial and selective coding (Merriam, 2002).The analysis involved a dynamic and flexible interaction with the data, "enter[ing] the investigation with an open mind, ready to hear what participants are saying," rather than a proceduralized sequence of steps (Corbin, 2009, p. 51).This iterative interaction, facilitated by NVivo12, allowed for participant voices to be projected and examined without restrictions of researcherdetermined codes, labels, or concepts.Intermediate qualitative findings prompted statistical hypothesis testing through non-parametric correlational analysis within the larger dataset, which was conducted using SPSS.Quantitizing was employed for data transformation (Sandelowski et al., 2009) to represent frequencies and prominence of the emerged themes.
The first step of interview analysis involved grouping transcripts in two categories, with 8 interviews offering episodic references to the CLB (e.g., "It [PBLA] has made me work with the CLBs more, which is a good thing" [T49]), and 21 transcripts including elaborated references (e.g., "What is challenging?…theknowledge of the CLB benchmarks.... I feel that we all should have been more engaged in the CLB and the knowledge of, you know, what is a benchmark 2 in writing, what is a benchmark 3.Although we had many … workshops on it, you never have enough … I find the CLB a challenge" [T106]).In total, 84 participants provided either an open-ended comment or interview references to the CLB, whereas 13 teachers offered both.The range and depth of unprompted CLB-related discussions by participants demonstrated the urgency of documenting practitioner experiences in this dedicated article, particularly considering the dearth of such literature.

Findings
Before closer examination of specific themes that emerged in the process of grounded theory analysis of qualitative data, Tables 1 and 2 present the themes from survey comments with their frequencies and sample quotes illustrating concerns about the CLB and its use.   1) were less frequent than the other two themes, Learner knowledge of the CLB and Questioning learner understanding of the CLB (Table 2).As reflected in Table 1, of the 69 participants, nine teachers appreciated the CLB, while 14 problematized an aspect of CLB use, with a frequent focus on variable interpretations of the standard (N=11, n=16).According to Table 2, while 22 teachers commented on learner knowledge of the CLB, 48 teachers questioned learner understanding of the language standard.In response to RQ1 (How do LINC teachers describe their experiences with the CLB as part of PBLA implementation process?), the two tables demonstrated that participants' experiences with the CLB have been different from the celebratory tone of predominantly aspirational CLB literature.While appreciation of the CLB and its role was present, comments on challenges dominated the dataset.

LINC Students' Understanding of the CLB as Reported by Teachers
A summary of quantitative responses to the four survey questions is presented in Table 3. Teachers were more confident in students' knowledge of their individual benchmarks (Q11, 79.4% agree or strongly agree; 11.2% disagree or strongly disagree) than in students' understanding of what the CLBs are (Q10, 65.5% agree or strongly agree; 20.7% disagree or strongly disagree).Open-ended survey comments frequently problematized learners' experiences with the standard (Tables 1 and 2).The most common category in the smaller sub-theme of Learner knowledge of the CLB (n=24) was Learners Have a General Idea of the CLB (n=13): "I think they have an awareness of benchmarks, but I doubt they could tell you anything about the Canadian Language Benchmarks" (T42); "Now, after all this time, they do seem to understand them to some degree" (T233).However, teachers distinguished between knowing and understanding, frequently questioning the latter (n=69): "I'm not sure they [students] understand what they [benchmarks] mean as they're so convoluted..." (T1); "Most students have a grasp of what benchmark they are in, but not a grasp of what that actually means for them and their language ability" (T38).
Teachers observed that learners' understanding of the CLB did not always improve after extensive explanations: "Even though they [the benchmarks] have been discussed at length, it is hard for them [students] to wrap their heads around how these are used" (T196); "Stage One learners are bombarded with institutional forms along with receiving CLB information.It is very difficult to fully explain the concept of Canadian Language Benchmarks" (T207).Teachers suggested (N=4) that learner understanding may not reach deeper than numeric representations of levels: "To most, they are just numbers and [they] don't realize that each level has different tasks" (T61).

Differentiated Learner Understanding of Benchmarks Across CLB Levels
To quantitatively probe for a possible relationship between students' language levels and their understanding of the CLB, correlational analysis was conducted on the ordinal variables.Results of the Spearman correlation indicated a weak positive association between the CLB levels currently taught by a teacher (Q52) and perception of students' knowledge of the CLB (Q10), (rs(218)= .20,p< .003).The higher the CLB level the teacher taught, the more confident the teacher was in students' knowledge of the CLB.A similarly weak positive association was found between the CLB levels currently taught by a teacher (Q52) and the teacher's perception of students' knowledge of their individual benchmarks (Q11), (rs(219)= .21,p< .002).These findings were corroborated by comments on challenges in comprehending the CLB, their functions and purpose, particularly at beginner levels: "You can't expect people who are level 6-7 and Literacy and CLB 1 and 2 to be working at the same level of understanding and metacognition and reflection.It doesn't work that way" (T154).In the interview, T163 described their work with the CLB checklists, or Can Do statements (CCLB, n.d.)Even though learners seemed to have a better understanding of the benchmarks at higher levels, some teachers questioned the necessity of explicit CLB instruction recommended by the CCLB (2019c).T128 purposefully de-emphasized the CLB as numeric representations of learner progress: I'm not getting them to focus on their CLB levels, I'm getting them to focus on improving their own English … trying to improve for themselves, not for the CLB levels … For them, knowing in detail about CLB … it's all interesting, but it wouldn't drive them to learn faster or better.That's more for a teacher, so they will use all that information.T97 also questioned the numerical focus on benchmarks: … we had exercises when … training to be a teacher.They give you … an example of student work, okay, tell us what level is this person?… I think what we need to do is say, what is the objective of all of this?…I know, for citizenship you need to be at level 4, I understand that.But there is more to teaching ESL than trying to give somebody a number.

Appreciating the CLB: "Good, but It Took a Lot Just to Understand It"
Interviews revealed a critical teacher stance toward the CLB (26 out of 29 participants expressed concerns).Ten participants elaborated on challenges presented by the CLB itself, and 23 commented on implementation challenges, while 12 teachers expressed appreciation of the document.Note.N -number of teacher participants who commented on the theme/sub-theme in their interviews; n -number of coding references for the theme/sub-theme across the interviews; BMbenchmarks Of the 12 participants appreciating the CLB, 11 commented positively about their CLB training: "We had a whole-day seminar on placing … which level the student was in … how to tell what level that [sample] was in.We had our training around that, which I really liked" (T178).However, five teachers also expressed reservations about the training, in addition to appreciation: "[It] was hard because you were introduced to a lot of terms in a very static or kind of disembodied environment.You're learning about profiles of ability and performance indicators … all together.It was a challenge more than I thought it would be" (T202).T69 was direct in re-iterating the gap between the training and practical PBLA demands: … very little actual examples, like, how do you do it, and give me the exact test, and instead … all the criteria, you know, go to CLB document … look at this, look at that, how you create an assessment, and what it has to incorporate, like, look at…this textual knowledge, grammatical knowledge, sociolinguistic knowledge.… You know, when you're talking about, let's say a dialogue, a speaking assessment, which I had in class, taking your car for repair to the mechanic, and the conversation, and just the vocabulary.Do I need to do this… analysis in depth, what part of it is textual, what part is functional?… This is just spending a lot of time on all these things…and instead, just let's focus on practical side of it because the practical side is what everybody is now failing.Where do you get these…real-life tasks?Some of them you can find there but … a lot of theory, and a lot of justification, why we should do, and all this … very in-depth linguistic analysis of skills, which is not really needed, I think.

Problematizing the CLB: "Teachers are Just Trying to Interpret the Document"
In response to RQ1, most participants commented on challenges while interpreting and applying the CLB.Within the largest CLB-related interview theme, Problematizing the CLB, two distinct types of challenges surfaced, those stemming from 1) the document itself (N=10, n=32), and 2) the requirement to implement it (N=23, n=88), suggesting that while CLB application itself was not unproblematic, the challenges increased under PBLA pressures.Eighteen teachers (Table 4) offered examples of the CLB interpretation issues (n=60).With over 16 years of teaching experience, T210 from Manitoba, where the CLB had been promoted earlier than in other provinces, repeatedly referred to the lack of interpretation consistency, which resulted from the vagueness of the CLB language: I don't see a lot of…consistency with each teacher teaching the same level… Something says moderately, and then the next level is adequately or good... It's very…subjective.I know they are trying to that it's not subjective, but it still is subjective… I don't think it [consistency] is possible because they say moderately good at this.Every teacher's gonna have a good idea of what's moderate.They just change the adverb slightly from one level to the next.
Responding to a question about sources of students' confusion with an assessment designed by PBLA Leads, T210 explained that the confusion was caused by "the teachers … just trying to interpret the document." Similar mismatches between students' comprehension and printed governmentsponsored resources for their benchmarks were reported by T69: "[it was CLB] 3. Most of them failed.And I got it from … LINC 3 activity binder.So, with PBLA, with all this new way of assessing … their levels definitely dropped.What level 3 was before, was much higher as opposed to what it is now."Discouraged by such instances of students' failing a presumably benchmark-appropriate task, the teacher turned to the CLB document for guidance: [Turning pages and reading from the CLB] So here is my benchmark … So, uses appropriate polite expressions.… Speaks in short sentences with some evidence of connected discourse.Fluency is just barely adequate.Vocabulary is somewhat limited.When it's somewhat limited, is it 2, is it 1?Is it 3? [referring to the marking scheme in a rubric].70% is completing.[Laughs] ... All these scales, it's so [pause] arbitrary.
T47, when describing attempts to use the CLB, attributed challenges to … the vagaries that are in the CLB 2012 document.And the tasks that are suggested there are so difficult to assess.… Of course, I keep it [the CLB] by my desk because I keep looking at those tasks constantly [laughs], but, you know, 'Listen to a movie review of two movies and decide what movie to watch.' How do you assess that?If the student listens and says, I am going to the first movie, what?Do we have to have reasons for doing that?[Laughs.]So, I find that tasks that they suggest in the book are not particularly useful.
As a result, T4 reported frequently "hitting the level way above" the benchmark of their students.
As a possible solution, twelve interview participants suggested (n=28) a bank of CLB-aligned teaching and assessment resources, created for immediate use and/or referencing in the task design process: "we should have had the resources.So, not just the curriculum resources but those standardized rubrics, too, those assessments that tell us, what does it mean to say that someone has adequate control of grammar and spelling at this level?what does developing mean?" (T47).T140 insisted that the resources need to be multilevel, i.e., addressing a span of benchmarks, "because that's what usually the classroom looks like.So, understanding the design of the classroom" is necessary to address practical challenges productively.

"Unstandardized Standard"
With insufficient benchmarked resources, the most frequently observed controversy in the interviews was that the CLB became an "unstandardized standard" (N=12, n=24).According to T140, "there is a lot of variation in understanding.I'd been a recipient of binders from other centres that students have attended with multiple teachers, and when they end up in my classroom, it has been interesting.I think the standards, even though they are called standards, they are not quite standardized."T143 explained lack of consistency: "We are using different assessment tasks, we've got different standards, and as long as we have teachers and PBLA Leads trying to interpret this, and force teachers to donate time, to create their own assessment tasks and so on, we'll always, always have inconsistency."T40 reflected on the goals of the standardization process: What's the point of everyone creating their own assessments?I don't know.I just feel like not even within one institution [assessments are consistent], and of course not between institutions.I think it's a huge challenge that might need to be addressed at the national level probably.That's a whole system challenge.So, as far as what can be done to address that, I guess having a bank of assessments would be the first thing that comes to my mind.I just think this notion that we are creating [assessments] and they are magically going to be the same, you know, at the same level, makes no sense at all.T34 suggested that consistency may not be achievable or desirable due to the variability of local and individual contexts: "even in one city, a level 4 in one end of the city is very different from level 4 in socioeconomic in another."Two teachers suggested a standard curriculum and standardized assessments.T4 argued, "I personally prefer that we have standardized tests so that my level 1 is the same as a level 1 at another school."Such comments were frequent in the larger dataset on PBLA, without necessarily mentioning the CLB.
Among controversies of the CLB document and its implementation, T46 struggled to reconcile an apparent conflict between the two CLB principles -competency-based and learner-centred.T46 called the contradiction "a major theoretical flaw," which required students to master competencies even though such mastery may not always be relevant for individual learners.Other teacher-reported CLB limitations included the standard being "very weak on digital citizenship" (T47), as well as the "functional" focus of the document, which may discourage from in-depth study of broader issues beyond immediate settlement needs: … this has always been my criticism of the CLB, they are so functional.They are really based on just … getting things done, reproducing information, interacting with others, and sharing information, which is really what we do in communication, but they are so, so functional, they are based on day-to-day life.So, the students are developing, I think, a more insular view of the world because of PBLA and the CLBs.And I know the CLBs do talk about pragmatic competence, the sociocultural competence, intercultural competence, all of those things.But they sometimes get pushed to the side.
Concerns about the dominant focus on functions/tasks at the expense of either more transferrable underlying language skills or deeper understanding of the new culture and society were echoed by four other participants: We are doing things that are compartmentalized, so it's not continuous, it's faulty production line… There's a lack of grammar.It's too disjointed.Okay, real-world task, where does that leave you?Is that transferable?Where else could you take it?… Yes, doing so many tasks, achieving so many competencies.I sense in using it that this thing is disjointed, it's not a smooth-running thing (T46).
Language competencies, as articulated in the CLB (Interacting with Others, Giving/Comprehending Instructions, Reproducing Information, Getting Things Done, and Sharing/Comprehending Information) were mentioned as a source of confusion by seven participants.The reasons for questioning the competencies may stem from PBLA requirement to assess them with equal frequency, while teachers disagreed that all these competencies deserved equal attention: "I tend to make more tasks for sharing information because that to me is what we do the most, and reproducing information is pretty straightforward.…We spend more time on sharing information and the pragmatics of interacting with others" (T47).T42, from a college preparation program, was frustrated by the requirement of an equal number of assessments for all competencies: When you are assessing … ability to copy from the dictionary compared to their ability to write a paragraph, that's ludicrous.I don't mind teaching that particular skill, but actually having to assess their ability to copy something and hold that as an equal measure of an academic skill, that's wasting my students' time, that's wasting my time.
Teacher comments exemplified the need for the CLB concepts and expectations to be accessible, achievable, and relevant both at the theoretical and practical levels before practitioners are required to demonstrate routine achievement of those expectations.

Applying the CLB in PBLA
In response to RQ2 (How do LINC teachers see the relationships between the CLB and PBLA?), the data offered quantitative responses to Q31 (PBLA training has increased my familiarity with the CLB), as well as open-ended comments (N=22, n=62), ranging from inherent similarities between the two to the CLB being a precondition of PBLA success.For T210, "it's the same thing.They are kind of together," while T32 saw PBLA guiding CLB implementation.T154 elaborated (n=6) on the necessary fluency with the CLB and TBLT prior to PBLA: "If you understand what CLBs are and if you've been doing taskbased all these years anyway, PBLA is not that much of a leap, it's not that different.… The CLB and the task-based is the cake, and PBLA is the icing on the cake."Survey comments ranged from apparently neutral -"It is necessary to understand both [CLB and PBLA] as they work together" (T233), to indications of tension -"I was already working closely with the CLBs.If this [PBLA] was a means of making instructors use the CLBs, surely there would have been easier ways to accomplish that?" (T168).
Quantitatively, the majority of participants (66.5%) either agreed or strongly agreed that their PBLA teacher training increased their CLB knowledge (Q31), while 16.5% disagreed, 10.3% strongly disagreed, and 6.7% were not sure (Table 3).Those who disagreed reported previous knowledge of the document (n=17): "…well familiar with CLB before PBLA" (T122).A range in CLB knowledge and application between LINC service provider organizations (SPOs) was observed: "my organization has always been dedicated to extensive CLB professional development … other organizations in town, instructors had to learn about the CLBs to implement PBLA" (T154).
Correlational analysis explored the relationships between the length of teaching experience and response to Q31. Results of Spearman correlation indicated a weak negative association between reporting increased CLB knowledge caused by PBLA training and the length of teaching language to adults, (rs(219)= -.23, p< .001).The longer their teaching experience, the less frequently the participants reported an increase in CLB knowledge after PBLA teacher training.It suggested that, if teaching experience had contributed to CLB knowledge, PBLA teacher training may not have been effective in further increases.
Those who agreed that PBLA training increased their familiarity with the CLB offered a variety of comments on the outcomes, from apparently positive -"I think I refer to the CLB and competencies more" (T158), to disappointed ones -"I now know just how flawed and vague that CLB document really is" (T20).T1, who was "not sure" about Q31, remarked that knowledge of the CLB "shouldn't be the point of teaching."Open-ended comments elaborated on the challenges that CLB concepts posed for learners (n=58), but similar challenges for teachers were also reported (n=16).As T185 stated when referring to PBLA requirements to provide students with frequent CLB-aligned assessments, "especially when combined with the vague-ness of the clbs and a lack of graded materials, I (and my colleagues) are always second-guessing our tasks." The challenges of delineating whether issues raised by participants were directly related to the CLB or stemming from PBLA protocol were illustrated by the interview with T32.These excerpts should be read in the context of PBLA implementation at the SPO, which was featured as a contrasting case in a study on a segment of the larger dataset (Desyatova, 2018).PBLA implementation practices utilized at this SPO were particularly alarming because LINC students could be retained at the same benchmark level due to failed assessments, interpreted as teacher failure in creating CLB-aligned PBLA-compliant assessments.This questionable practice of punishing learners for teachers' purported inability to accurately follow the CLB document and PBLA guidelines contributed to the following comments: … we have the CLB, and we read it to the best of our ability and our understanding … we are all teaching with different qualifications.Some people have the innate ability to understand a document as it was presented to be understood, … they can understand things to a deeper level.Maybe they get CLB and it's fast and easy, and everything they can create is perfect.But then there are others of us who do the best we can do, right?… We are blindly believing that we are professionals, and we are blindly believing that we can read, we are blindly believing that we have a brain in our head, and we are blindly believing that we've read the document, and we think we know what a real-world task is, … and so we create things going along, and then damn, somebody at the end can say, you are an idiot.Weird.This is a bad idea.Well, that is just so unkind and ridiculous.
Other interview comments on the CLB implementation in PBLA (N=22, n=62) painted a less gloomy picture, and centred around two major sub-themes: 1) PBLA as a tool for promoting the CLB (N=6, n=10) and TBLT (N=3, n=9); and 2) CLB knowledge as a necessary precondition of PBLA implementation (N=4, n=11).Quantitatively, 54.3% of respondents still found it difficult to create reliable CLB-based assessments for their students (Q38), while 37.5% disagreed with the statement, and 8.2% were unsure (Table 3).Results of Spearman correlation demonstrated a very weak negative association between the length of teaching language to adults (Q49) and teachers' continued difficulty in creating CLB-based assessments (Q38), (rs(214) =-.14, p< .046).Therefore, the longer teachers had been teaching language to adults, the less difficulty they reported in creating CLB-based assessments.No association was found between the length of PBLA implementation (Q55) and teachers' difficulty in creating CLB-based assessments (Q38).Similarly, no correlation was discovered between teachers' difficulty in creating CLBbased assessments (Q38) and teacher's age (Q57).The findings challenged the assumptions that PBLA implementation would increase teacher fluency in productive CLB application, as well as assumptions that older teachers found it difficult to learn how to use the CLB because they "resist change" (T238).
An important sub-theme that emerged within the theme of interactions between the CLB and PBLA was a potential neglect of the CLB principle of learner-centredness in PBLA (N=14, n=22).This idea was almost unanimously articulated as PBLA requirements and resources being too taxing and unsuitable for beginners (N=13, n=21).

Discussion: CLB Tensions and Contradictions
In response to RQ1, some teachers reported productive experiences with the CLB either pre-or post-PBLA.However, most participants observed either contradictions in the document or challenges with its implementation.In response to RQ2, connections between the CLB and PBLA were another major theme emerging from the grounded-theory-driven analysis of teacher comments.Answering RQ3 (how the relationships between the CLB and PBLA moderate teacher response to the CLB), it was concluded that PBLA introduced additional pressures on practitioners, while the CLB document itself was not unproblematic prior to PBLA.
Quantitative surveys attempted to capture practitioner experiences with the CLB, but such measures represented a surface-level response to the Likert-type items, which may not have reflected the teachers' lived experiences.While percentages in Table 3 could be read as successful CLB uptake, it is participants' interpretations, provided in qualitative data, that should be considered representative of the CLB uptake.A close examination of teachers' comments revealed issues that became more pressing due to repeatedly documented PBLA challenges and contradictions.
Improving language teaching and learning must have been a goal of the CLB as a language standard.However, questions arise whether the goal can be achieved by enforcing the CLB through PBLA as a questionable assessment protocol.Within the context of this study, it was not always feasible to delineate which benefits and challenges were attributable to the CLB document itself, and which arose from its interaction with PBLA as an implementation tool.The difficulty partly stemmed from the goals of the larger project to examine PBLA implementation.However, similar challenges would persist in other studies of CLB application due to the contextual reality of the CLB being the key component in mandatory PBLA implementation.At the same time, teacher experiences with the CLB and its application, documented in this study, can illuminate both strengths and weaknesses of the standard, irrespective of PBLA.
While LINC students may demonstrate the knowledge of the CLB on quantitative measures, most teachers questioned the feasibility of ensuring an adequate understanding of the CLB on the part of the learners, particularly beginners.Even though verbal unpacking of the concept of benchmarks was repeatedly attempted by teachers, they doubted learners could comprehend and benefit from metacognitive elaborations in a new language.Similar to earlier studies (Abbott, 2019;Fox & Fraser, 2012;Karasova, 2019), literacy learners were observed to be struggling with comprehending both the CLB and PBLA.
Pushing for an increased learner awareness of the CLB in PBLA, regardless of language levels (CCLB, 2019a(CCLB, , 2019c(CCLB, , 2019d, 2019e), 2019e), may be signalling an underlying contradiction between the declared purposes of the standard and the blanket scope of its mandatory dissemination.Fine-grained distinctions between the 12 benchmarks of language proficiency, from complete beginners to proficient users are at the heart of the language standard both conceptually and physically, taking up 144 pages of the 235 total pages in the document (CCLB, 2012).However, PBLA insisted on communicating CLB concepts to all LINC learners, including beginners.If Benchmark 1 describes a language level where learners can "recognize letters and numbers, a small number of words, and very short, simple phrases related to everyday objects and immediate needs" (CCLB, 2012, p. 74), metacognitive descriptions of language progression can not be comprehensively articulated at this level, albeit in simplified written and pictorial language of the Can Do Statements (CCLB, n.d.).It appears that efforts to disseminate the CLB among beginner LINC students disregarded the benchmarks themselves as accurate descriptors of language proficiency.Whether neglecting limited language proficiency can still produce a learnercentred environment, which is one of the declared CLB principles, is the most pressing question arising from this investigation.
Another fundamental challenge experienced by teachers was variable interpretations ensuing from attempts to translate the standard into CLB-aligned classroom tasks and assessments.According to participants, the divergent interpretations undermined the language standard mission of the CLB, which was explicitly emphasized both in the document and PBLA as a tool for ensuring consistency in assessment (CCLB, 2017(CCLB, , 2019a;;Pettis, 2015).While more teaching and assessment resources are presently available (e.g., www.tutela.ca,www.realworldtasks.ca),at the initial stages of PBLA implementation, individual teachers were expected to create multiple assessment instruments for their classes, which resulted in dramatic increases in teacher workload, stress, and attrition (Desyatova, 2018(Desyatova, , 2020;;Fox & Fraser, 2012;Mohammadian, 2016;Ripley, 2018).The claim that difficulties experienced by teachers in CLB use and PBLA implementation were a sign of growing pains and would dissipate in due course (CCLB, 2017) was challenged by correlational analysis in this study, which demonstrated that increased length of PBLA implementation did not result in increased ease of CLB-aligned material creation for the participants.
The findings demonstrate that the translation of the CLB into curriculum and teaching materials is time-consuming, labour-intensive, and needs to be thoughtfully sustained, rather than forced on already disadvantaged and constrained teachers.The expectations that teachers would be able to create/source CLB-aligned materials with PBLA-required frequency contradicted the need for LINC-appropriate resources that practitioners repeatedly articulated (CIC, 2010;Haque & Valeo, 2017;Smit & Turcot, 2010).The findings also challenge the informally articulated assumptions that older teachers may be relying on traditional teaching approaches and experiencing difficulties with the CLB/PBLA due to "fear of change" (CCLB, 2017) and unwillingness to modify their presumably outdated, i.e., non-task-based, teaching.
Probably the most contradictory tension between the CLB document and its implementation through PBLA was the claim of consistent standards operationalized through classroom-based assessment, which was presented in PBLA as an alternative to "standardized testing" (CCLB, 2017(CCLB, , 2019a;;Pettis, 2015).Ironically, after experiencing multiple inconsistencies combined with the top-down pressures for standardization, teachers started leaning towards the standardized testing idea, probably with the goal of outsourcing the excessive workload and stress that they were subjected to when attempting to translate the CLB into frequent and consistent assessments.It appears that the system's pressure for standardization eclipsed the CLB principle of learner-centredness.In the forced, labour-intensive, and frequently unproductive attempts to produce consistent application of the CLB in PBLA, teachers started searching for a more manageable means of meeting the demand for consistency.For some participants, standard curriculum with standardized assessment tasks appeared to be a solution.The concomitant neglect of learner-centredness in such standardization was not explicitly expressed by teachers in this study, but considering the diversity of LINC learners, standardizing teaching and assessment may be at odds with the necessary "flexibility within programs to meet varied needs of clients" (Derwing, 2017, p. 88).The tensions between the goals of standardization and flexibility in meeting the diverse needs of learners and communities need to be examined in future studies.The neglect of practitioner challenges in implementing the CLB through PBLA resulted in teachers prioritizing their need to ease the pressures on individual practitioners through standard curriculum and materials, rather than continued search for learner-oriented solutions.
Such outcomes contradicting the intended goals in the application of a language standard have been documented with language proficiency frameworks in other contexts.Piccardo et al. (2019) described a variety of tools to support translation of the complex vision of a language standard into practice, concluding that "it seems important to have multiple approaches, rather than seeing one single tool or technique as a panacea" (p.120).In applying educational standards, the tendency for powerful system needs to override individual learner needs has been well-documented (Brindley, 1998 as cited by Llosa, 2011).Jang (2014) highlighted the potential for irreconcilable tensions when contradictory policy aspirations are combined in one assessment tool.It appears that PBLA, while ensuring CLB use, violated the CLB principle of learner-centredness, despite declared intentions to promote it.

Conclusions and Implications
This article complements academic literature on the CLB with practitioners' experiences with the language standard during the mandatory PBLA implementation, which aggravated CLB interpretation challenges.In the phenomenological research tradition, the study highlights the lived experiences of teacher practitioners as an essential part of the phenomenon itself.The article aims to bring "deeper insight to help us [e.g., applied linguists and policy makers] be in greater contact with the world" (Usher & Jackson, 2014, p. 3), the lived world of LINC teachers as CLB intended users.
The scope of the study did not allow to examine other controversies that teachers observed in the CLB document, such as rigid separation into four traditional skills, insufficient attention to developing digital literacy and language accuracy, tensions between learner-centred and competency-based CLB principles, as well as the limitations of purely functional language learning, potentially leading to "an insular view of the world" (T47), which had been problematized by Canadian scholars (Clark et al. 2012;Cray & Currie, 2004;Fleming, 2010Fleming, , 2015)).
The most significant limitation of this study was that data collection tools were not initially designed to examine teacher experiences with the CLB and its implementation, which must have resulted in frequent omissions of relevant data collection opportunities.Participants were not explicitly invited to elaborate on the CLB during the interviews, and only four Likert survey items mentioned the standard.However, the large group of participants and open-ended response opportunities still produced a rich dataset creating windows into practitioner experiences.The study reinforces the need to consider lived experiences when evaluating policy innovations, rather than relying on aspirational claims and neglecting feedback that does not conform to intended outcomes.Further research on the CLB needs to include observational studies of intact classrooms, which would allow for the triangulation of data sources.
The issues that were examined with sufficient details demonstrated that productive links are required between the CLB document and LINC teachers as the standard implementers, and PBLA has failed to become such a productive link.Given the practitioners' challenges stemming from the document itself, the CLB may not be effectively promoting its key principle of learner-centredness through PBLA as a mandatory tool.If this principle is of true importance to funders and policy makers, then the impact of PBLA on the application of the CLB as a language standard needs to be attended to.Without fundamental revisions to PBLA, both the practitioners' perspectives and the CLB principles risk being marginalized-to the detriment of the LINC program and its outcomes-and the CLB reduced to what Fulcher (2010) described as: … the real interest of many policy makers lies in using levels and descriptors to set minimum levels of achievement that help them to hold institutions and teachers accountable for delivering the outcomes specified as essential in their own policies.The standards-based systems provide the rationale for the infrastructure of controls that micromanage the behaviour of learners, teachers and institutions (p.14).
Findings of this study suggest that PBLA has become such an "infrastructure of control and micromanagement," rather than facilitating the CLB potential to inspire improvements and innovation in language teaching.Whether the CLB could have realized this potential under different conditions is a question for further research.
While the CLB as a language standard has been an important step in the LINC program development, extra care is needed when selecting tools for the implementation of the standard, and even more so prior to making the tool(s) mandatory.The findings demonstrated that instead of facilitating CLB uptake by practitioners, PBLA has aggravated challenges experienced by teachers when they were required to apply the standard to teaching and assessment frequently and effectively.When envisioning and enacting beneficial links between policy and practice, it is imperative to engage both academic and practitioner communities.This will help avoid the compounding negative effects of questionable initiatives declared mandatory on a national scale, as has been the case with PBLA.Most alarmingly, PBLA might be undermining the key CLB principle of learner-centredness, despite having claimed otherwise.

Table 1
Frequencies of major themes, sub-themes, and sample quotes from the CLB-related survey comments(N=69, n=126)

Table 2
Frequencies of major themes, sub-themes, and sample quotes from the CLB-related survey Note.N -number of teacher participants who commented on the theme/sub-theme in the survey; n -number of CLB-related coding references in survey comments; T -teacher participant (e.g., T42); italics were used for participant quotes.The tables summarize four major themes (two per table) where Appreciating the CLB and Problematizing CLB use (Table

Table 3
Teacher responses to Likert-type survey items, percentage Q10.My students know what Canadian Language Benchmarks are Q11.My students know what their individual Canadian Language Benchmarks are Q31.PBLA training has increased my familiarity with the CLB Q38.I still find it difficult to create reliable CLB-based assessment instruments for my students , in a CLB 3 class: … they are supposed to go through each CLB, each part and check what they already know … that's hard for them … because they can read a sentence about what they should be able to accomplish but they don't really have a mental picture of what that looks like.So, I feel like it takes me two hours to go through it all, if they actually understand what they are checking [laughs], if I actually want them to do it well.They don't grasp that vocabulary necessarily, 'cause they don't really know … how to check if they can or can't do it, at what level can they or can't do it [laughs].It's so subjective that it's almost pointless to me (T163).
Table 4 presents frequencies of categorized interview comments.